[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:38:49 dist-test-slave-z47j krb5kdc[12278](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:38:49 dist-test-slave-z47j krb5kdc[12278](info): set up 2 sockets
Sep 27 18:38:49 dist-test-slave-z47j krb5kdc[12278](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260927 18:38:50.146915 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:38:50 dist-test-slave-z47j krb5kdc[12278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:38:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:38:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:38:50.286757 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:36167
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:36699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:36167
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:50.374418 12291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:50.374686 12291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:50.374725 12291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:50.375921 12291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:38:50.375991 12291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:50.376019 12291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:50.376050 12291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:38:50.376075 12291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:38:50.377673 12291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:36699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:36167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:36167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:38:50.377960 12291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:50.378206 12291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:38:50.384672 12291 server_base.cc:1034] running on GCE node
W20260927 18:38:50.384646 12297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:50.384640 12296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:50.384646 12299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:50.385324 12291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:50.385773 12291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:50.386981 12291 hybrid_clock.cc:648] HybridClock initialized: now 1790534330386957 us; error 42 us; skew 500 ppm
Sep 27 18:38:50 dist-test-slave-z47j krb5kdc[12278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534330, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:50.398922 12291 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:38:50.399616 12291 webserver.cc:460] Webserver started at http://127.11.251.254:43663/ using document root <none> and password file <none>
I20260927 18:38:50.399962 12291 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:50.400049 12291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:50.400247 12291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:50.401690 12291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "7ab309443231437fb4c5ada14526d7ed"
format_stamp: "Formatted at 2026-09-27 18:38:50 on dist-test-slave-z47j"
server_key: "b35686ff96d7859db9b2db32995bbc72"
server_key_iv: "f27eb7d1cdb5f5a60a8183b3a353cb0f"
server_key_version: "encryptionkey@0"
I20260927 18:38:50.402158 12291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "7ab309443231437fb4c5ada14526d7ed"
format_stamp: "Formatted at 2026-09-27 18:38:50 on dist-test-slave-z47j"
server_key: "b35686ff96d7859db9b2db32995bbc72"
server_key_iv: "f27eb7d1cdb5f5a60a8183b3a353cb0f"
server_key_version: "encryptionkey@0"
I20260927 18:38:50.404078 12291 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260927 18:38:50.405162 12306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:50.405443 12291 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:50.405601 12291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "7ab309443231437fb4c5ada14526d7ed"
format_stamp: "Formatted at 2026-09-27 18:38:50 on dist-test-slave-z47j"
server_key: "b35686ff96d7859db9b2db32995bbc72"
server_key_iv: "f27eb7d1cdb5f5a60a8183b3a353cb0f"
server_key_version: "encryptionkey@0"
I20260927 18:38:50.405679 12291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:50.425992 12291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:50.428681 12291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:50.428871 12291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:50.434265 12291 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:36167
I20260927 18:38:50.434334 12358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:36167 every 8 connection(s)
I20260927 18:38:50.434789 12291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:38:50.435885 12359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:50.438558 12359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed: Bootstrap starting.
I20260927 18:38:50.439196 12359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:50.439520 12359 log.cc:824] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:50.440375 12359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed: No bootstrap required, opened a new log
I20260927 18:38:50.442152 12359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab309443231437fb4c5ada14526d7ed" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 36167 } }
I20260927 18:38:50.442296 12359 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:50.442319 12359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ab309443231437fb4c5ada14526d7ed, State: Initialized, Role: FOLLOWER
I20260927 18:38:50.442505 12359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab309443231437fb4c5ada14526d7ed" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 36167 } }
I20260927 18:38:50.442575 12359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:38:50.442600 12359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:38:50.442680 12359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:50.443223 12359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab309443231437fb4c5ada14526d7ed" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 36167 } }
I20260927 18:38:50.443349 12359 leader_election.cc:302] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ab309443231437fb4c5ada14526d7ed; no voters:
I20260927 18:38:50.443495 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 12291
I20260927 18:38:50.443574 12359 leader_election.cc:288] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:38:50.443603 12362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:50.443651 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:38:50.443719 12362 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed [term 1 LEADER]: Becoming Leader. State: Replica: 7ab309443231437fb4c5ada14526d7ed, State: Running, Role: LEADER
I20260927 18:38:50.443795 12362 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab309443231437fb4c5ada14526d7ed" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 36167 } }
I20260927 18:38:50.443835 12359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:38:50.444247 12363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ab309443231437fb4c5ada14526d7ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab309443231437fb4c5ada14526d7ed" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 36167 } } }
I20260927 18:38:50.444454 12363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:50.444622 12271 external_mini_cluster.cc:1428] Setting key 997cacd5bcfdafb79398f118b3719658
I20260927 18:38:50.444901 12366 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:38:50.445127 12364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ab309443231437fb4c5ada14526d7ed. Latest consensus state: current_term: 1 leader_uuid: "7ab309443231437fb4c5ada14526d7ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab309443231437fb4c5ada14526d7ed" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 36167 } } }
I20260927 18:38:50.445254 12364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:50.445876 12366 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:38:50.448038 12366 catalog_manager.cc:1347] Generated new cluster ID: 6d18b02b48184f7e862ef726ead9a7c1
I20260927 18:38:50.448096 12366 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:38:50 dist-test-slave-z47j krb5kdc[12278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:38:50.461251 12366 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:38:50.462049 12366 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:38:50.466351 12381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:50.446300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35660 (local address 127.11.251.254:36167)
0927 18:38:50.446517 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:50.446528 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:50.446732 (+ 204us) server_negotiation.cc:407] Connection header received
0927 18:38:50.449162 (+ 2430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:50.449172 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:50.449250 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:50.449425 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:50.451932 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:50.453604 (+ 1672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:50.455355 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:50.456070 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:50.456187 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:50.461418 (+ 5231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:50.461425 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:50.461425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:50.461443 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:50.463883 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:50.464529 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:50.464532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:50.464533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:50.464656 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:50.464978 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:50.464979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:50.464979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:50.465319 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:50.465503 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:50.465887 (+ 384us) server_negotiation.cc:299] Negotiation successful
0927 18:38:50.466083 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":101,"threads_started":1}
I20260927 18:38:50.474421 12366 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ab309443231437fb4c5ada14526d7ed: Generated new TSK 0
I20260927 18:38:50.474655 12366 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.251.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
I20260927 18:38:50.505240 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:36167
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:36699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:50.612449 12385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:50.612777 12385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:50.612814 12385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:50.614315 12385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:50.614357 12385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:50.614389 12385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:38:50.615971 12385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:36699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:36167
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:38:50.616222 12385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:50.616442 12385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:50.622404 12390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:50.622435 12391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:50.622448 12393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:50.622728 12385 server_base.cc:1034] running on GCE node
I20260927 18:38:50.622995 12385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:50.623296 12385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:50.624480 12385 hybrid_clock.cc:648] HybridClock initialized: now 1790534330624445 us; error 57 us; skew 500 ppm
Sep 27 18:38:50 dist-test-slave-z47j krb5kdc[12278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534330, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:50.627470 12385 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:38:50.628041 12385 webserver.cc:460] Webserver started at http://127.11.251.193:42499/ using document root <none> and password file <none>
I20260927 18:38:50.628317 12385 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:50.628391 12385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:50.628553 12385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:50.629869 12385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "213c02dc9eb446c38c9b7cff22ebd09e"
format_stamp: "Formatted at 2026-09-27 18:38:50 on dist-test-slave-z47j"
server_key: "a9ecc63882b9b000ef582ed2999dd037"
server_key_iv: "e945df885d2fa6f5d0c322a0139ff5fa"
server_key_version: "encryptionkey@0"
I20260927 18:38:50.630281 12385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "213c02dc9eb446c38c9b7cff22ebd09e"
format_stamp: "Formatted at 2026-09-27 18:38:50 on dist-test-slave-z47j"
server_key: "a9ecc63882b9b000ef582ed2999dd037"
server_key_iv: "e945df885d2fa6f5d0c322a0139ff5fa"
server_key_version: "encryptionkey@0"
I20260927 18:38:50.631816 12385 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:38:50.632759 12400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:50.633044 12385 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:50.633173 12385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "213c02dc9eb446c38c9b7cff22ebd09e"
format_stamp: "Formatted at 2026-09-27 18:38:50 on dist-test-slave-z47j"
server_key: "a9ecc63882b9b000ef582ed2999dd037"
server_key_iv: "e945df885d2fa6f5d0c322a0139ff5fa"
server_key_version: "encryptionkey@0"
I20260927 18:38:50.633271 12385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:50.658684 12385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:50.661382 12385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:50.661597 12385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:50.661921 12385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:50.662364 12385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:50.662421 12385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:50.662458 12385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:50.662487 12385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:50.670271 12385 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:35453
I20260927 18:38:50.670365 12513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:35453 every 8 connection(s)
I20260927 18:38:50.670771 12385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:38:50.671923 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 12385
I20260927 18:38:50.672047 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:38:50.672238 12271 external_mini_cluster.cc:1428] Setting key 83c6ec12a8939a2ac57204f8b3b7fa1d
Sep 27 18:38:50 dist-test-slave-z47j krb5kdc[12278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534330, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.251.194@KRBTEST.COM; defaulting to no policy
I20260927 18:38:50.682936 12381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:50.671691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:51217 (local address 127.11.251.254:36167)
0927 18:38:50.671751 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:50.671753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:50.672022 (+ 269us) server_negotiation.cc:407] Connection header received
0927 18:38:50.672764 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:50.672770 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:50.672780 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:50.672874 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:50.674113 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:50.674951 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:50.676259 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:50.676507 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:50.676591 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:50.679552 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:50.679554 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:50.679554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:50.679564 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:50.681286 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:50.681855 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:50.681856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:50.681857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:50.681919 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:50.682211 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:50.682212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:50.682212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:50.682340 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:50.682414 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:50.682745 (+ 331us) server_negotiation.cc:299] Negotiation successful
0927 18:38:50.682839 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:38:50.683156 12516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:50.671634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:36167 (local address 127.11.251.193:51217)
0927 18:38:50.671856 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:50.671905 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:50.672643 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:50.672976 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:50.672981 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:50.673780 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:50.673998 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:50.674004 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:50.675078 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:50.675080 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:50.676143 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:50.676147 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:50.676617 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:50.676618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:50.676789 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:50.677417 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:50.677429 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:50.679445 (+ 2016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:50.681410 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:50.681417 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:50.681420 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:50.681763 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:50.681995 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:50.681996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:50.681996 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:50.682136 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:50.682414 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:50.682422 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:50.682604 (+ 182us) client_negotiation.cc:769] Sending connection context
0927 18:38:50.682711 (+ 107us) client_negotiation.cc:240] Negotiation successful
0927 18:38:50.682896 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":134,"threads_started":1}
I20260927 18:38:50.683686 12514 heartbeater.cc:347] Connected to a master server at 127.11.251.254:36167
I20260927 18:38:50.683791 12514 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:50.683948 12514 heartbeater.cc:511] Master 127.11.251.254:36167 requested a full tablet report, sending...
I20260927 18:38:50.684554 12323 ts_manager.cc:194] Registered new tserver with Master: 213c02dc9eb446c38c9b7cff22ebd09e (127.11.251.193:35453)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.194@KRBTEST.COM" created.
I20260927 18:38:50.685374 12323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:51217
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
I20260927 18:38:50.702135 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:36167
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:36699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:50.790284 12519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:50.790514 12519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:50.790539 12519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:50.791733 12519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:50.791782 12519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:50.791810 12519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:38:50.793339 12519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:36699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:36167
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:38:50.793672 12519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:50.793926 12519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:50.799512 12524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:50.799549 12527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:50.799535 12525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:50.799811 12519 server_base.cc:1034] running on GCE node
I20260927 18:38:50.800092 12519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:50.800383 12519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:50.801559 12519 hybrid_clock.cc:648] HybridClock initialized: now 1790534330801551 us; error 37 us; skew 500 ppm
Sep 27 18:38:50 dist-test-slave-z47j krb5kdc[12278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534330, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:50.804643 12519 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:38:50.805141 12519 webserver.cc:460] Webserver started at http://127.11.251.194:38531/ using document root <none> and password file <none>
I20260927 18:38:50.805351 12519 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:50.805399 12519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:50.805491 12519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:50.806602 12519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "60fc2ec72a154645a7c1dac39482f194"
format_stamp: "Formatted at 2026-09-27 18:38:50 on dist-test-slave-z47j"
server_key: "f67425a92dfba392d61b7b61242b5b80"
server_key_iv: "efd0f41cce3427145e0efc6f02a45931"
server_key_version: "encryptionkey@0"
I20260927 18:38:50.806907 12519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "60fc2ec72a154645a7c1dac39482f194"
format_stamp: "Formatted at 2026-09-27 18:38:50 on dist-test-slave-z47j"
server_key: "f67425a92dfba392d61b7b61242b5b80"
server_key_iv: "efd0f41cce3427145e0efc6f02a45931"
server_key_version: "encryptionkey@0"
I20260927 18:38:50.808271 12519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:38:50.809221 12534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:50.809455 12519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:50.809566 12519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "60fc2ec72a154645a7c1dac39482f194"
format_stamp: "Formatted at 2026-09-27 18:38:50 on dist-test-slave-z47j"
server_key: "f67425a92dfba392d61b7b61242b5b80"
server_key_iv: "efd0f41cce3427145e0efc6f02a45931"
server_key_version: "encryptionkey@0"
I20260927 18:38:50.809746 12519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:50.819883 12519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:50.822461 12519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:50.822587 12519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:50.822804 12519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:50.823172 12519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:50.823210 12519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:50.823235 12519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:50.823289 12519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:50.831135 12519 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:33779
I20260927 18:38:50.831250 12647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:33779 every 8 connection(s)
I20260927 18:38:50.831588 12519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:38:50.838301 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 12519
I20260927 18:38:50.838397 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:38:50.838600 12271 external_mini_cluster.cc:1428] Setting key dc5e0f8307d189b8fc31514b0e0171aa
Sep 27 18:38:50 dist-test-slave-z47j krb5kdc[12278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534330, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:38:50.842557 12381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:50.832471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:44151 (local address 127.11.251.254:36167)
0927 18:38:50.832557 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:50.832560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:50.832847 (+ 287us) server_negotiation.cc:407] Connection header received
0927 18:38:50.833489 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:50.833491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:50.833503 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:50.833617 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:50.834651 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:50.835204 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:50.836349 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:50.836574 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:50.836635 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:50.839153 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:50.839154 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:50.839155 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:50.839163 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:50.840865 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:50.841349 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:50.841350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:50.841351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:50.841424 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:50.841749 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:50.841750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:50.841751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:50.841902 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:50.841983 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:50.842328 (+ 345us) server_negotiation.cc:299] Negotiation successful
0927 18:38:50.842415 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:38:50.842554 12650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:50.832442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:36167 (local address 127.11.251.194:44151)
0927 18:38:50.832682 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:50.832726 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:50.833358 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:50.833704 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:50.833708 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:50.834324 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:50.834552 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:50.834556 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:50.835318 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:50.835319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:50.836194 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:50.836196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:50.836677 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:50.836678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:50.836793 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:50.837328 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:50.837337 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:50.839056 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:50.840972 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:50.840980 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:50.840983 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:50.841234 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:50.841506 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:50.841506 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:50.841507 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:50.841677 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:50.842019 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:50.842025 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:50.842144 (+ 119us) client_negotiation.cc:769] Sending connection context
0927 18:38:50.842253 (+ 109us) client_negotiation.cc:240] Negotiation successful
0927 18:38:50.842378 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":109,"threads_started":1}
I20260927 18:38:50.843242 12648 heartbeater.cc:347] Connected to a master server at 127.11.251.254:36167
I20260927 18:38:50.843361 12648 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:50.843611 12648 heartbeater.cc:511] Master 127.11.251.254:36167 requested a full tablet report, sending...
I20260927 18:38:50.844071 12323 ts_manager.cc:194] Registered new tserver with Master: 60fc2ec72a154645a7c1dac39482f194 (127.11.251.194:33779)
I20260927 18:38:50.844609 12323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:44151
WARNING: no policy specified for kudu/127.11.251.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
I20260927 18:38:50.872792 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:36167
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:36699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:50.967031 12653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:50.967314 12653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:50.967352 12653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:50.968621 12653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:50.968699 12653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:50.968748 12653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:38:50.970453 12653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:36699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:36167
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:38:50.970770 12653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:50.971040 12653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:50.977512 12658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:50.977532 12659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:50.977690 12653 server_base.cc:1034] running on GCE node
W20260927 18:38:50.977532 12661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:50.978106 12653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:50.978403 12653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:50.979547 12653 hybrid_clock.cc:648] HybridClock initialized: now 1790534330979536 us; error 39 us; skew 500 ppm
Sep 27 18:38:50 dist-test-slave-z47j krb5kdc[12278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534330, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:50.982529 12653 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:38:50.983024 12653 webserver.cc:460] Webserver started at http://127.11.251.195:39903/ using document root <none> and password file <none>
I20260927 18:38:50.983296 12653 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:50.983367 12653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:50.983521 12653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:50.984704 12653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "ac5e16d6ffcd468ba4c012b7b6689044"
format_stamp: "Formatted at 2026-09-27 18:38:50 on dist-test-slave-z47j"
server_key: "89b85479dd65c60704444b74842a1d5d"
server_key_iv: "82bd46e2f454fcfe7720d0063c237759"
server_key_version: "encryptionkey@0"
I20260927 18:38:50.985100 12653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "ac5e16d6ffcd468ba4c012b7b6689044"
format_stamp: "Formatted at 2026-09-27 18:38:50 on dist-test-slave-z47j"
server_key: "89b85479dd65c60704444b74842a1d5d"
server_key_iv: "82bd46e2f454fcfe7720d0063c237759"
server_key_version: "encryptionkey@0"
I20260927 18:38:50.986660 12653 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:38:50.987546 12668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:50.987849 12653 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:38:50.987975 12653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "ac5e16d6ffcd468ba4c012b7b6689044"
format_stamp: "Formatted at 2026-09-27 18:38:50 on dist-test-slave-z47j"
server_key: "89b85479dd65c60704444b74842a1d5d"
server_key_iv: "82bd46e2f454fcfe7720d0063c237759"
server_key_version: "encryptionkey@0"
I20260927 18:38:50.988070 12653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:51.004590 12653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:51.007038 12653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:51.007220 12653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:51.007469 12653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:51.007810 12653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:51.007874 12653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:51.007912 12653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:51.007962 12653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:51.015651 12653 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:40159
I20260927 18:38:51.015704 12781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:40159 every 8 connection(s)
I20260927 18:38:51.016031 12653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:38:51.018901 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 12653
I20260927 18:38:51.019013 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:38:51.019201 12271 external_mini_cluster.cc:1428] Setting key a3927e53f74fec2d2e6e615eae003777
Sep 27 18:38:51 dist-test-slave-z47j krb5kdc[12278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534330, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:38:51.027380 12381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:51.016902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:33467 (local address 127.11.251.254:36167)
0927 18:38:51.016977 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:51.016980 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:51.017148 (+ 168us) server_negotiation.cc:407] Connection header received
0927 18:38:51.017667 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:51.017668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:51.017682 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:51.017789 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:51.018786 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:51.019580 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:51.021035 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:51.021306 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:51.021397 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:51.024089 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:51.024091 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:51.024091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:51.024107 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:51.025988 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:51.026497 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:51.026498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:51.026499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:51.026551 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:51.026815 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:51.026816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:51.026816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:51.026929 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:51.026989 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:51.027237 (+ 248us) server_negotiation.cc:299] Negotiation successful
0927 18:38:51.027312 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:38:51.027624 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:51.016844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:36167 (local address 127.11.251.195:33467)
0927 18:38:51.017004 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:51.017046 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:51.017541 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:51.017878 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:51.017882 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:51.018480 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:51.018677 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:51.018682 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:51.019723 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:51.019724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:51.020918 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:51.020920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:51.021403 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:51.021404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:51.021504 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:51.022076 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:51.022085 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:51.023921 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:51.026092 (+ 2171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:51.026099 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:51.026102 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:51.026406 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:51.026621 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:51.026621 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:51.026622 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:51.026752 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:51.026978 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:51.026981 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:51.027116 (+ 135us) client_negotiation.cc:769] Sending connection context
0927 18:38:51.027208 (+ 92us) client_negotiation.cc:240] Negotiation successful
0927 18:38:51.027354 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260927 18:38:51.028148 12782 heartbeater.cc:347] Connected to a master server at 127.11.251.254:36167
I20260927 18:38:51.028272 12782 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:51.028466 12782 heartbeater.cc:511] Master 127.11.251.254:36167 requested a full tablet report, sending...
I20260927 18:38:51.028954 12323 ts_manager.cc:194] Registered new tserver with Master: ac5e16d6ffcd468ba4c012b7b6689044 (127.11.251.195:40159)
I20260927 18:38:51.029312 12323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:33467
I20260927 18:38:51.031554 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 18:38:51 dist-test-slave-z47j krb5kdc[12278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534331, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 18:38:51 dist-test-slave-z47j krb5kdc[12278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534331, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.251.193@KRBTEST.COM
I20260927 18:38:51.058318 12789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:51.049793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46932 (local address 127.11.251.193:35453)
0927 18:38:51.049983 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:51.049988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:51.050008 (+ 20us) server_negotiation.cc:407] Connection header received
0927 18:38:51.050043 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:51.050045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:51.050059 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:51.050141 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:51.050790 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:51.051495 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:51.052346 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:51.052667 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:51.052728 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:51.055313 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:51.055320 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:51.055320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:51.055328 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:51.056963 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:51.057329 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:51.057332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:51.057333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:51.057395 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:51.057705 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:51.057706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:51.057706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:51.057843 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:51.058014 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:51.058160 (+ 146us) server_negotiation.cc:299] Negotiation successful
0927 18:38:51.058234 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
W20260927 18:38:51.058853 12428 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46932
Sep 27 18:38:51 dist-test-slave-z47j krb5kdc[12278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 18:38:51 dist-test-slave-z47j krb5kdc[12278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.193@KRBTEST.COM
I20260927 18:38:51.083760 12789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:51.075989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46936 (local address 127.11.251.193:35453)
0927 18:38:51.076107 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:51.076111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:51.076245 (+ 134us) server_negotiation.cc:407] Connection header received
0927 18:38:51.076348 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:51.076349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:51.076373 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:51.076451 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:51.077102 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:51.077752 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:51.078550 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:51.078776 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:51.078856 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:51.081318 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:51.081320 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:51.081321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:51.081331 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:51.082725 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:51.083088 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:51.083089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:51.083089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:51.083139 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:51.083331 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:51.083331 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:51.083331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:51.083435 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:51.083534 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:51.083618 (+ 84us) server_negotiation.cc:299] Negotiation successful
0927 18:38:51.083692 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260927 18:38:51.084619 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 12385
I20260927 18:38:51.089741 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 12519
I20260927 18:38:51.096889 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 12653
I20260927 18:38:51.102401 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 12291
2026-09-27T18:38:51Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2011 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:38:51 dist-test-slave-z47j krb5kdc[12796](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:38:51 dist-test-slave-z47j krb5kdc[12796](info): set up 2 sockets
Sep 27 18:38:51 dist-test-slave-z47j krb5kdc[12796](info): commencing operation
krb5kdc: starting...
W20260927 18:38:52.165750 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:38:52 dist-test-slave-z47j krb5kdc[12796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:38:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:38:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:38:52.294524 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:40421
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:42673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:40421
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:52.385070 12809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:52.385358 12809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:52.385397 12809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:52.386698 12809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:38:52.386775 12809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:52.386798 12809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:52.386819 12809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:38:52.386850 12809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:38:52.388424 12809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:42673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:40421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:40421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:38:52.388758 12809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:52.389016 12809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:52.394444 12815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:52.394467 12814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:52.394467 12817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:52.394666 12809 server_base.cc:1034] running on GCE node
I20260927 18:38:52.395118 12809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:52.395483 12809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:52.396639 12809 hybrid_clock.cc:648] HybridClock initialized: now 1790534332396620 us; error 39 us; skew 500 ppm
Sep 27 18:38:52 dist-test-slave-z47j krb5kdc[12796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534332, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:52.399415 12809 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:38:52.399879 12809 webserver.cc:460] Webserver started at http://127.11.251.254:38193/ using document root <none> and password file <none>
I20260927 18:38:52.400135 12809 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:52.400187 12809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:52.400329 12809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:52.401476 12809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "9abf94f8e4db45cca8534de7d2ae2e94"
format_stamp: "Formatted at 2026-09-27 18:38:52 on dist-test-slave-z47j"
server_key: "0d7fb59f308066fa81c304ecd6aea3c9"
server_key_iv: "6035dc969581c1eed53a3d0cf3097b81"
server_key_version: "encryptionkey@0"
I20260927 18:38:52.401923 12809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "9abf94f8e4db45cca8534de7d2ae2e94"
format_stamp: "Formatted at 2026-09-27 18:38:52 on dist-test-slave-z47j"
server_key: "0d7fb59f308066fa81c304ecd6aea3c9"
server_key_iv: "6035dc969581c1eed53a3d0cf3097b81"
server_key_version: "encryptionkey@0"
I20260927 18:38:52.403401 12809 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:52.404304 12824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:52.404592 12809 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:52.404703 12809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "9abf94f8e4db45cca8534de7d2ae2e94"
format_stamp: "Formatted at 2026-09-27 18:38:52 on dist-test-slave-z47j"
server_key: "0d7fb59f308066fa81c304ecd6aea3c9"
server_key_iv: "6035dc969581c1eed53a3d0cf3097b81"
server_key_version: "encryptionkey@0"
I20260927 18:38:52.404784 12809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:52.413992 12809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:52.416182 12809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:52.416339 12809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:52.420678 12809 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:40421
I20260927 18:38:52.420758 12876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:40421 every 8 connection(s)
I20260927 18:38:52.421115 12809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:38:52.422266 12877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:52.424932 12877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94: Bootstrap starting.
I20260927 18:38:52.425683 12877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:52.426020 12877 log.cc:824] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:52.426872 12877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94: No bootstrap required, opened a new log
I20260927 18:38:52.428776 12877 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9abf94f8e4db45cca8534de7d2ae2e94" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40421 } }
I20260927 18:38:52.428942 12877 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:52.428967 12877 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9abf94f8e4db45cca8534de7d2ae2e94, State: Initialized, Role: FOLLOWER
I20260927 18:38:52.429165 12877 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9abf94f8e4db45cca8534de7d2ae2e94" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40421 } }
I20260927 18:38:52.429253 12877 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:38:52.429277 12877 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:38:52.429359 12877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:52.430013 12877 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9abf94f8e4db45cca8534de7d2ae2e94" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40421 } }
I20260927 18:38:52.430114 12877 leader_election.cc:302] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9abf94f8e4db45cca8534de7d2ae2e94; no voters:
I20260927 18:38:52.430191 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 12809
I20260927 18:38:52.430333 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:38:52.430392 12877 leader_election.cc:288] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:38:52.430424 12880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:52.430526 12271 external_mini_cluster.cc:1428] Setting key 27559fb51aaa4cd0abe92ec6fc8489e3
I20260927 18:38:52.430620 12880 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94 [term 1 LEADER]: Becoming Leader. State: Replica: 9abf94f8e4db45cca8534de7d2ae2e94, State: Running, Role: LEADER
I20260927 18:38:52.430747 12880 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9abf94f8e4db45cca8534de7d2ae2e94" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40421 } }
I20260927 18:38:52.430783 12877 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:38:52.431362 12880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9abf94f8e4db45cca8534de7d2ae2e94. Latest consensus state: current_term: 1 leader_uuid: "9abf94f8e4db45cca8534de7d2ae2e94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9abf94f8e4db45cca8534de7d2ae2e94" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40421 } } }
I20260927 18:38:52.431586 12880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:52.431546 12881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9abf94f8e4db45cca8534de7d2ae2e94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9abf94f8e4db45cca8534de7d2ae2e94" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40421 } } }
I20260927 18:38:52.431797 12881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:52.432698 12891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:38:52.433315 12891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:38:52.435420 12891 catalog_manager.cc:1347] Generated new cluster ID: 7c14900548a8439dbf1b25af4ca4668b
I20260927 18:38:52.435482 12891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:38:52 dist-test-slave-z47j krb5kdc[12796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:38:52.443674 12884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:52.431040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60652 (local address 127.11.251.254:40421)
0927 18:38:52.431353 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:52.431366 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:52.431415 (+ 49us) server_negotiation.cc:407] Connection header received
0927 18:38:52.431587 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:52.431595 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:52.431672 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:52.432656 (+ 984us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:52.432825 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:52.434306 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:52.435433 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:52.436083 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:52.436216 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:52.439503 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:52.439507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:52.439508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:52.439522 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:52.441813 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:52.442272 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:52.442275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:52.442276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:52.442395 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:52.442651 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:52.442651 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:52.442652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:52.442982 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:52.443143 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:52.443299 (+ 156us) server_negotiation.cc:299] Negotiation successful
0927 18:38:52.443451 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":77,"threads_started":1}
I20260927 18:38:52.460225 12891 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:38:52.461066 12891 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:38:52.465282 12891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9abf94f8e4db45cca8534de7d2ae2e94: Generated new TSK 0
I20260927 18:38:52.465497 12891 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.251.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
I20260927 18:38:52.496835 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:40421
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:42673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:52.589741 12903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:52.590041 12903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:52.590081 12903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:52.591357 12903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:52.591426 12903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:52.591476 12903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:38:52.593055 12903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:42673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:40421
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:38:52.593353 12903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:52.593659 12903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:52.599311 12908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:52.599355 12909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:52.599332 12911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:52.599376 12903 server_base.cc:1034] running on GCE node
I20260927 18:38:52.599845 12903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:52.600158 12903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:52.601333 12903 hybrid_clock.cc:648] HybridClock initialized: now 1790534332601313 us; error 48 us; skew 500 ppm
Sep 27 18:38:52 dist-test-slave-z47j krb5kdc[12796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534332, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:52.604112 12903 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:38:52.604602 12903 webserver.cc:460] Webserver started at http://127.11.251.193:39465/ using document root <none> and password file <none>
I20260927 18:38:52.604848 12903 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:52.604918 12903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:52.605080 12903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:52.606266 12903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "28528f946190408ab095c1201d592c5f"
format_stamp: "Formatted at 2026-09-27 18:38:52 on dist-test-slave-z47j"
server_key: "ec0723e9c94f5976c2d895440e5993ae"
server_key_iv: "9745ca926ec6568786adccccb8506554"
server_key_version: "encryptionkey@0"
I20260927 18:38:52.606647 12903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "28528f946190408ab095c1201d592c5f"
format_stamp: "Formatted at 2026-09-27 18:38:52 on dist-test-slave-z47j"
server_key: "ec0723e9c94f5976c2d895440e5993ae"
server_key_iv: "9745ca926ec6568786adccccb8506554"
server_key_version: "encryptionkey@0"
I20260927 18:38:52.608179 12903 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:38:52.609000 12918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:52.609263 12903 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:52.609372 12903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "28528f946190408ab095c1201d592c5f"
format_stamp: "Formatted at 2026-09-27 18:38:52 on dist-test-slave-z47j"
server_key: "ec0723e9c94f5976c2d895440e5993ae"
server_key_iv: "9745ca926ec6568786adccccb8506554"
server_key_version: "encryptionkey@0"
I20260927 18:38:52.609462 12903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:52.632354 12903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:52.634616 12903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:52.634788 12903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:52.635056 12903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:52.635419 12903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:52.635478 12903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:52.635514 12903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:52.635543 12903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:52.642277 12903 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:43435
I20260927 18:38:52.642411 13031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:43435 every 8 connection(s)
I20260927 18:38:52.642729 12903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:38:52.643204 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 12903
I20260927 18:38:52.643316 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:38:52.643504 12271 external_mini_cluster.cc:1428] Setting key c62d09c3e365735ce8f2bf6e2473b984
Sep 27 18:38:52 dist-test-slave-z47j krb5kdc[12796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534332, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.251.194@KRBTEST.COM; defaulting to no policy
I20260927 18:38:52.655679 12884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:52.643835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:42455 (local address 127.11.251.254:40421)
0927 18:38:52.643902 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:52.643905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:52.644035 (+ 130us) server_negotiation.cc:407] Connection header received
0927 18:38:52.645509 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:52.645510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:52.645525 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:52.645666 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:52.646728 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:52.647564 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:52.649284 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:52.649498 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:52.649667 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:52.652521 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:52.652523 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:52.652523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:52.652530 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:52.654086 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:52.654556 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:52.654557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:52.654557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:52.654618 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:52.654918 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:52.654919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:52.654919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:52.655025 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:52.655101 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:52.655492 (+ 391us) server_negotiation.cc:299] Negotiation successful
0927 18:38:52.655579 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 18:38:52.655887 13034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:52.643656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:40421 (local address 127.11.251.193:42455)
0927 18:38:52.643883 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:52.643939 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:52.645363 (+ 1424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:52.645771 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:52.645776 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:52.646247 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:52.646605 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:52.646611 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:52.647693 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:52.647695 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:52.649161 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:52.649165 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:52.649688 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:52.649691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:52.649839 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:52.650552 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:52.650561 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:52.652401 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:52.654190 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:52.654197 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:52.654200 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:52.654474 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:52.654692 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:52.654692 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:52.654693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:52.654803 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:52.655103 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:52.655112 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:52.655309 (+ 197us) client_negotiation.cc:769] Sending connection context
0927 18:38:52.655471 (+ 162us) client_negotiation.cc:240] Negotiation successful
0927 18:38:52.655651 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":101,"threads_started":1}
I20260927 18:38:52.656555 13032 heartbeater.cc:347] Connected to a master server at 127.11.251.254:40421
I20260927 18:38:52.656652 13032 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.194@KRBTEST.COM" created.
I20260927 18:38:52.656852 13032 heartbeater.cc:511] Master 127.11.251.254:40421 requested a full tablet report, sending...
I20260927 18:38:52.657657 12841 ts_manager.cc:194] Registered new tserver with Master: 28528f946190408ab095c1201d592c5f (127.11.251.193:43435)
I20260927 18:38:52.658428 12841 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:42455
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
I20260927 18:38:52.673305 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:40421
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:42673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:52.767671 13037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:52.767923 13037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:52.767951 13037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:52.769173 13037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:52.769210 13037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:52.769238 13037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:38:52.770828 13037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:42673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:40421
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:38:52.771052 13037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:52.771265 13037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:52.776741 13045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:52.776741 13042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:52.776760 13043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:52.777031 13037 server_base.cc:1034] running on GCE node
I20260927 18:38:52.777304 13037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:52.777632 13037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:52.778831 13037 hybrid_clock.cc:648] HybridClock initialized: now 1790534332778809 us; error 44 us; skew 500 ppm
Sep 27 18:38:52 dist-test-slave-z47j krb5kdc[12796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534332, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:52.782437 13037 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:38:52.783052 13037 webserver.cc:460] Webserver started at http://127.11.251.194:39211/ using document root <none> and password file <none>
I20260927 18:38:52.783322 13037 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:52.783401 13037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:52.783566 13037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:52.784801 13037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "fb0ee83254084df2833fbd736002d333"
format_stamp: "Formatted at 2026-09-27 18:38:52 on dist-test-slave-z47j"
server_key: "0598421ac240f3ff5a2ca4035137974d"
server_key_iv: "ea4be777cae61d025d07ab1edc3f4884"
server_key_version: "encryptionkey@0"
I20260927 18:38:52.785235 13037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "fb0ee83254084df2833fbd736002d333"
format_stamp: "Formatted at 2026-09-27 18:38:52 on dist-test-slave-z47j"
server_key: "0598421ac240f3ff5a2ca4035137974d"
server_key_iv: "ea4be777cae61d025d07ab1edc3f4884"
server_key_version: "encryptionkey@0"
I20260927 18:38:52.786885 13037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260927 18:38:52.787774 13052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:52.788039 13037 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:52.788157 13037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "fb0ee83254084df2833fbd736002d333"
format_stamp: "Formatted at 2026-09-27 18:38:52 on dist-test-slave-z47j"
server_key: "0598421ac240f3ff5a2ca4035137974d"
server_key_iv: "ea4be777cae61d025d07ab1edc3f4884"
server_key_version: "encryptionkey@0"
I20260927 18:38:52.788244 13037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:52.808735 13037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:52.811127 13037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:52.811311 13037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:52.811612 13037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:52.811976 13037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:52.812032 13037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:52.812083 13037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:52.812107 13037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:52.819801 13037 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:39091
I20260927 18:38:52.819908 13165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:39091 every 8 connection(s)
I20260927 18:38:52.820261 13037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
Sep 27 18:38:52 dist-test-slave-z47j krb5kdc[12796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534332, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:38:52.829149 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 13037
I20260927 18:38:52.829259 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:38:52.829447 12271 external_mini_cluster.cc:1428] Setting key 2fb26830e86ad9d570068e297b1dbd67
I20260927 18:38:52.831125 12884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:52.821130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:32983 (local address 127.11.251.254:40421)
0927 18:38:52.821237 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:52.821240 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:52.821509 (+ 269us) server_negotiation.cc:407] Connection header received
0927 18:38:52.822601 (+ 1092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:52.822602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:52.822621 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:52.822684 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:52.823361 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:52.823909 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:52.825013 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:52.825233 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:52.825316 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:52.827759 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:52.827761 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:52.827761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:52.827767 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:52.829153 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:52.829853 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:52.829854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:52.829854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:52.829919 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:52.830237 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:52.830238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:52.830238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:52.830397 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:52.830477 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:52.830865 (+ 388us) server_negotiation.cc:299] Negotiation successful
0927 18:38:52.830963 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260927 18:38:52.831130 13168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:52.821097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:40421 (local address 127.11.251.194:32983)
0927 18:38:52.821348 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:52.821391 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:52.822484 (+ 1093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:52.822757 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:52.822761 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:52.823054 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:52.823277 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:52.823280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:52.824027 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:52.824029 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:52.824911 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:52.824914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:52.825326 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:52.825327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:52.825456 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:52.826043 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:52.826052 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:52.827642 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:52.829301 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:52.829310 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:52.829313 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:52.829749 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:52.829998 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:52.829998 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:52.829999 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:52.830140 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:52.830479 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:52.830492 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:52.830677 (+ 185us) client_negotiation.cc:769] Sending connection context
0927 18:38:52.830776 (+ 99us) client_negotiation.cc:240] Negotiation successful
0927 18:38:52.830929 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":140,"threads_started":1}
I20260927 18:38:52.831652 13166 heartbeater.cc:347] Connected to a master server at 127.11.251.254:40421
I20260927 18:38:52.831775 13166 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:52.832027 13166 heartbeater.cc:511] Master 127.11.251.254:40421 requested a full tablet report, sending...
I20260927 18:38:52.832556 12841 ts_manager.cc:194] Registered new tserver with Master: fb0ee83254084df2833fbd736002d333 (127.11.251.194:39091)
I20260927 18:38:52.832990 12841 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:32983
WARNING: no policy specified for kudu/127.11.251.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
I20260927 18:38:52.857472 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:40421
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:42673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:52.943768 13171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:52.943984 13171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:52.944008 13171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:52.945276 13171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:52.945325 13171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:52.945353 13171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:38:52.946887 13171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:42673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:40421
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:38:52.947137 13171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:52.947398 13171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:52.952741 13179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:52.952751 13176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:52.952881 13177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:52.952893 13171 server_base.cc:1034] running on GCE node
I20260927 18:38:52.953310 13171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:52.953634 13171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:52.954828 13171 hybrid_clock.cc:648] HybridClock initialized: now 1790534332954795 us; error 53 us; skew 500 ppm
Sep 27 18:38:52 dist-test-slave-z47j krb5kdc[12796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534332, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:52.957564 13171 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:38:52.958078 13171 webserver.cc:460] Webserver started at http://127.11.251.195:45803/ using document root <none> and password file <none>
I20260927 18:38:52.958330 13171 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:52.958421 13171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:52.958571 13171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:52.959636 13171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "cb987313423143cda7324b6e212b32f4"
format_stamp: "Formatted at 2026-09-27 18:38:52 on dist-test-slave-z47j"
server_key: "c4b6f09775a91e32d851e0565c350f8c"
server_key_iv: "c5c45fedc1be5289aae92dfce1868ab4"
server_key_version: "encryptionkey@0"
I20260927 18:38:52.960005 13171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "cb987313423143cda7324b6e212b32f4"
format_stamp: "Formatted at 2026-09-27 18:38:52 on dist-test-slave-z47j"
server_key: "c4b6f09775a91e32d851e0565c350f8c"
server_key_iv: "c5c45fedc1be5289aae92dfce1868ab4"
server_key_version: "encryptionkey@0"
I20260927 18:38:52.961485 13171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260927 18:38:52.962378 13186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:52.962639 13171 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:38:52.962750 13171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "cb987313423143cda7324b6e212b32f4"
format_stamp: "Formatted at 2026-09-27 18:38:52 on dist-test-slave-z47j"
server_key: "c4b6f09775a91e32d851e0565c350f8c"
server_key_iv: "c5c45fedc1be5289aae92dfce1868ab4"
server_key_version: "encryptionkey@0"
I20260927 18:38:52.962831 13171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:52.988942 13171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:52.991793 13171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:52.992015 13171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:52.992336 13171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:52.992719 13171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:52.992784 13171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:52.992831 13171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:52.992861 13171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:53.001392 13171 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:44809
I20260927 18:38:53.001441 13299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:44809 every 8 connection(s)
I20260927 18:38:53.001886 13171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:38:53.003680 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 13171
I20260927 18:38:53.003813 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:38:53.003988 12271 external_mini_cluster.cc:1428] Setting key ee9cdabd5f833418f27bca7c761f25a6
Sep 27 18:38:53 dist-test-slave-z47j krb5kdc[12796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534332, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:38:53.013969 12884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:53.002870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:37055 (local address 127.11.251.254:40421)
0927 18:38:53.002964 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:53.002967 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:53.003238 (+ 271us) server_negotiation.cc:407] Connection header received
0927 18:38:53.004430 (+ 1192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:53.004431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:53.004443 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:53.004533 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:53.005478 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.006101 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.007277 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.007540 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.007638 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:53.010605 (+ 2967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:53.010607 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:53.010607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:53.010615 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:53.012218 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:53.012644 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:53.012645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:53.012646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:53.012773 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:53.013140 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:53.013141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:53.013142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:53.013282 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:53.013376 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:53.013794 (+ 418us) server_negotiation.cc:299] Negotiation successful
0927 18:38:53.013889 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 18:38:53.014051 13302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:53.002813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:40421 (local address 127.11.251.195:37055)
0927 18:38:53.003026 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:53.003072 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:53.004288 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:53.004625 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:53.004630 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:53.005062 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:53.005368 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:53.005373 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.006196 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.006197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:53.007048 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:53.007049 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.007689 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.007690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:53.007820 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:53.008444 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:53.008452 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:53.010486 (+ 2034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:53.012309 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:53.012314 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:53.012316 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:53.012547 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:53.012903 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:53.012904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:53.012904 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:53.013043 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:53.013389 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:53.013396 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:53.013525 (+ 129us) client_negotiation.cc:769] Sending connection context
0927 18:38:53.013727 (+ 202us) client_negotiation.cc:240] Negotiation successful
0927 18:38:53.013863 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":97,"threads_started":1}
I20260927 18:38:53.014884 13300 heartbeater.cc:347] Connected to a master server at 127.11.251.254:40421
I20260927 18:38:53.014986 13300 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:53.015141 13300 heartbeater.cc:511] Master 127.11.251.254:40421 requested a full tablet report, sending...
I20260927 18:38:53.015657 12841 ts_manager.cc:194] Registered new tserver with Master: cb987313423143cda7324b6e212b32f4 (127.11.251.195:44809)
I20260927 18:38:53.016297 12841 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:37055
I20260927 18:38:53.016707 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:38:53.025480 12884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:53.018744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60662 (local address 127.11.251.254:40421)
0927 18:38:53.018845 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:53.018848 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:53.018917 (+ 69us) server_negotiation.cc:407] Connection header received
0927 18:38:53.019002 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:53.019003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:53.019014 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:53.019089 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:53.019698 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.020205 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.020949 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.021150 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.021239 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:53.022686 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:53.022687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:53.022688 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:53.022695 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:53.024210 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:53.024576 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:53.024578 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:53.024578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:53.024642 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:53.024870 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:53.024871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:53.024871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:53.025044 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:53.025154 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:53.025265 (+ 111us) server_negotiation.cc:299] Negotiation successful
0927 18:38:53.025351 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260927 18:38:53.028636 12840 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:38:53.030470 12840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:38:53.037949 13314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:53.033477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35928 (local address 127.11.251.194:39091)
0927 18:38:53.033984 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:53.033989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:53.033998 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:38:53.034043 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:53.034045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:53.034058 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:53.034148 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:53.034406 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.035228 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.036820 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.037550 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.037685 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:53.037749 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:53.037827 (+ 78us) server_negotiation.cc:299] Negotiation successful
0927 18:38:53.037869 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":137,"threads_started":1}
I20260927 18:38:53.037914 13311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:53.033387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:44809 (local address 127.0.0.1:44206)
0927 18:38:53.033754 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:53.033768 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:53.033840 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:53.034165 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:53.034168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:53.034171 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:53.034273 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:53.034276 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.035600 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.035601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:53.036715 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:53.036716 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.037509 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.037510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:53.037624 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:53.037650 (+ 26us) client_negotiation.cc:769] Sending connection context
0927 18:38:53.037723 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 18:38:53.037801 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":141,"threads_started":1}
I20260927 18:38:53.038246 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:53.033723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44206 (local address 127.11.251.195:44809)
0927 18:38:53.033979 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:53.033984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:53.033993 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:38:53.034032 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:53.034034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:53.034048 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:53.034160 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:53.034363 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.035473 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.036816 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.037414 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.037514 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:53.037577 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:53.038087 (+ 510us) server_negotiation.cc:299] Negotiation successful
0927 18:38:53.038151 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":133,"threads_started":1}
I20260927 18:38:53.038326 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:53.033768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59180 (local address 127.11.251.193:43435)
0927 18:38:53.034361 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:53.034365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:53.034375 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:38:53.034413 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:53.034415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:53.034431 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:53.034506 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:53.034754 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.035763 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.036983 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.037751 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.037820 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:53.037865 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:53.038215 (+ 350us) server_negotiation.cc:299] Negotiation successful
0927 18:38:53.038255 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":101,"threads_started":1}
I20260927 18:38:53.038758 13312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:53.033397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:43435 (local address 127.0.0.1:59180)
0927 18:38:53.033877 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:53.033887 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:53.033932 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:53.034585 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:53.034586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:53.034588 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:53.034651 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:53.034654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.035877 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.035878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:53.036881 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:53.036883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.037905 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.037906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:53.037992 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:53.037999 (+ 7us) client_negotiation.cc:769] Sending connection context
0927 18:38:53.038107 (+ 108us) client_negotiation.cc:240] Negotiation successful
0927 18:38:53.038170 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":136,"threads_started":1}
I20260927 18:38:53.037914 13313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:53.033380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:39091 (local address 127.0.0.1:35928)
0927 18:38:53.033630 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:53.033673 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:53.033748 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:53.034225 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:53.034226 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:53.034229 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:53.034307 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:53.034310 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.035328 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.035329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:53.036712 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:53.036716 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.037681 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.037682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:53.037749 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:53.037751 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:38:53.037799 (+ 48us) client_negotiation.cc:240] Negotiation successful
0927 18:38:53.037838 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":41,"thread_start_us":67,"threads_started":1}
I20260927 18:38:53.039175 13234 tablet_service.cc:1467] Processing CreateTablet for tablet 98190c3088bc48e38c43911fe993b889 (DEFAULT_TABLE table=test-table [id=01a4c06798ed46fe9c78b1a92e2badc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:53.039587 13234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98190c3088bc48e38c43911fe993b889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:53.039654 12966 tablet_service.cc:1467] Processing CreateTablet for tablet 98190c3088bc48e38c43911fe993b889 (DEFAULT_TABLE table=test-table [id=01a4c06798ed46fe9c78b1a92e2badc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:53.040027 12966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98190c3088bc48e38c43911fe993b889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:53.042275 13100 tablet_service.cc:1467] Processing CreateTablet for tablet 98190c3088bc48e38c43911fe993b889 (DEFAULT_TABLE table=test-table [id=01a4c06798ed46fe9c78b1a92e2badc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:53.042541 13317 tablet_bootstrap.cc:492] T 98190c3088bc48e38c43911fe993b889 P cb987313423143cda7324b6e212b32f4: Bootstrap starting.
I20260927 18:38:53.042635 13100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98190c3088bc48e38c43911fe993b889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:53.043213 13317 tablet_bootstrap.cc:654] T 98190c3088bc48e38c43911fe993b889 P cb987313423143cda7324b6e212b32f4: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:53.043340 13318 tablet_bootstrap.cc:492] T 98190c3088bc48e38c43911fe993b889 P 28528f946190408ab095c1201d592c5f: Bootstrap starting.
I20260927 18:38:53.043787 13317 log.cc:824] T 98190c3088bc48e38c43911fe993b889 P cb987313423143cda7324b6e212b32f4: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:53.044054 13318 tablet_bootstrap.cc:654] T 98190c3088bc48e38c43911fe993b889 P 28528f946190408ab095c1201d592c5f: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:53.044579 13318 log.cc:824] T 98190c3088bc48e38c43911fe993b889 P 28528f946190408ab095c1201d592c5f: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:53.044701 13317 tablet_bootstrap.cc:492] T 98190c3088bc48e38c43911fe993b889 P cb987313423143cda7324b6e212b32f4: No bootstrap required, opened a new log
I20260927 18:38:53.044765 13317 ts_tablet_manager.cc:1397] T 98190c3088bc48e38c43911fe993b889 P cb987313423143cda7324b6e212b32f4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:53.045406 13318 tablet_bootstrap.cc:492] T 98190c3088bc48e38c43911fe993b889 P 28528f946190408ab095c1201d592c5f: No bootstrap required, opened a new log
I20260927 18:38:53.045463 13318 ts_tablet_manager.cc:1397] T 98190c3088bc48e38c43911fe993b889 P 28528f946190408ab095c1201d592c5f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:53.046917 13321 tablet_bootstrap.cc:492] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333: Bootstrap starting.
I20260927 18:38:53.047500 13321 tablet_bootstrap.cc:654] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:53.047508 13317 raft_consensus.cc:348] T 98190c3088bc48e38c43911fe993b889 P cb987313423143cda7324b6e212b32f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb987313423143cda7324b6e212b32f4" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44809 } } peers { permanent_uuid: "fb0ee83254084df2833fbd736002d333" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 39091 } } peers { permanent_uuid: "28528f946190408ab095c1201d592c5f" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43435 } }
I20260927 18:38:53.047508 13318 raft_consensus.cc:348] T 98190c3088bc48e38c43911fe993b889 P 28528f946190408ab095c1201d592c5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb987313423143cda7324b6e212b32f4" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44809 } } peers { permanent_uuid: "fb0ee83254084df2833fbd736002d333" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 39091 } } peers { permanent_uuid: "28528f946190408ab095c1201d592c5f" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43435 } }
I20260927 18:38:53.047701 13318 raft_consensus.cc:374] T 98190c3088bc48e38c43911fe993b889 P 28528f946190408ab095c1201d592c5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:53.047701 13317 raft_consensus.cc:374] T 98190c3088bc48e38c43911fe993b889 P cb987313423143cda7324b6e212b32f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:53.047742 13317 raft_consensus.cc:729] T 98190c3088bc48e38c43911fe993b889 P cb987313423143cda7324b6e212b32f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb987313423143cda7324b6e212b32f4, State: Initialized, Role: FOLLOWER
I20260927 18:38:53.047742 13318 raft_consensus.cc:729] T 98190c3088bc48e38c43911fe993b889 P 28528f946190408ab095c1201d592c5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28528f946190408ab095c1201d592c5f, State: Initialized, Role: FOLLOWER
I20260927 18:38:53.047889 13318 consensus_queue.cc:260] T 98190c3088bc48e38c43911fe993b889 P 28528f946190408ab095c1201d592c5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb987313423143cda7324b6e212b32f4" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44809 } } peers { permanent_uuid: "fb0ee83254084df2833fbd736002d333" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 39091 } } peers { permanent_uuid: "28528f946190408ab095c1201d592c5f" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43435 } }
I20260927 18:38:53.047889 13317 consensus_queue.cc:260] T 98190c3088bc48e38c43911fe993b889 P cb987313423143cda7324b6e212b32f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb987313423143cda7324b6e212b32f4" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44809 } } peers { permanent_uuid: "fb0ee83254084df2833fbd736002d333" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 39091 } } peers { permanent_uuid: "28528f946190408ab095c1201d592c5f" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43435 } }
I20260927 18:38:53.048126 13321 log.cc:824] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:53.048210 13317 ts_tablet_manager.cc:1428] T 98190c3088bc48e38c43911fe993b889 P cb987313423143cda7324b6e212b32f4: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260927 18:38:53.048216 13318 ts_tablet_manager.cc:1428] T 98190c3088bc48e38c43911fe993b889 P 28528f946190408ab095c1201d592c5f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:38:53.048342 13300 heartbeater.cc:503] Master 127.11.251.254:40421 was elected leader, sending a full tablet report...
I20260927 18:38:53.048404 13032 heartbeater.cc:503] Master 127.11.251.254:40421 was elected leader, sending a full tablet report...
I20260927 18:38:53.049013 13321 tablet_bootstrap.cc:492] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333: No bootstrap required, opened a new log
I20260927 18:38:53.049079 13321 ts_tablet_manager.cc:1397] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:53.051199 13321 raft_consensus.cc:348] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb987313423143cda7324b6e212b32f4" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44809 } } peers { permanent_uuid: "fb0ee83254084df2833fbd736002d333" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 39091 } } peers { permanent_uuid: "28528f946190408ab095c1201d592c5f" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43435 } }
I20260927 18:38:53.051345 13321 raft_consensus.cc:374] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:53.051367 13321 raft_consensus.cc:729] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb0ee83254084df2833fbd736002d333, State: Initialized, Role: FOLLOWER
I20260927 18:38:53.051484 13321 consensus_queue.cc:260] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb987313423143cda7324b6e212b32f4" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44809 } } peers { permanent_uuid: "fb0ee83254084df2833fbd736002d333" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 39091 } } peers { permanent_uuid: "28528f946190408ab095c1201d592c5f" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43435 } }
I20260927 18:38:53.051790 13166 heartbeater.cc:503] Master 127.11.251.254:40421 was elected leader, sending a full tablet report...
I20260927 18:38:53.051791 13321 ts_tablet_manager.cc:1428] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260927 18:38:53.070894 13167 tablet.cc:2369] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:38:53.143657 13033 tablet.cc:2369] T 98190c3088bc48e38c43911fe993b889 P 28528f946190408ab095c1201d592c5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:38:53.252645 13301 tablet.cc:2369] T 98190c3088bc48e38c43911fe993b889 P cb987313423143cda7324b6e212b32f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:38:53.323163 13325 raft_consensus.cc:482] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:38:53.323341 13325 raft_consensus.cc:504] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb987313423143cda7324b6e212b32f4" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44809 } } peers { permanent_uuid: "fb0ee83254084df2833fbd736002d333" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 39091 } } peers { permanent_uuid: "28528f946190408ab095c1201d592c5f" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43435 } }
I20260927 18:38:53.323907 13325 leader_election.cc:288] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb987313423143cda7324b6e212b32f4 (127.11.251.195:44809), 28528f946190408ab095c1201d592c5f (127.11.251.193:43435)
I20260927 18:38:53.328418 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:53.324267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:58461 (local address 127.11.251.193:43435)
0927 18:38:53.324338 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:53.324340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:53.324349 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:38:53.324387 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:53.324388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:53.324401 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:53.324494 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:53.324742 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.325548 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.327217 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.327870 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.327974 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:53.327987 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:53.328254 (+ 267us) server_negotiation.cc:299] Negotiation successful
0927 18:38:53.328333 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 18:38:53.328428 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:53.324267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:42409 (local address 127.11.251.195:44809)
0927 18:38:53.324324 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:53.324327 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:53.324342 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:38:53.324414 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:53.324415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:53.324424 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:53.324495 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:53.324742 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.325548 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.327261 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.327908 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.327984 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:53.327993 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:53.328314 (+ 321us) server_negotiation.cc:299] Negotiation successful
0927 18:38:53.328359 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:38:53.328400 13168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:53.324135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:43435 (local address 127.11.251.194:58461)
0927 18:38:53.324199 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:53.324212 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:53.324259 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:53.324533 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:53.324534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:53.324535 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:53.324630 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:53.324632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.325667 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.325668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:53.327087 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:53.327090 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.328024 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.328025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:53.328143 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:53.328149 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:38:53.328245 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 18:38:53.328278 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":89}
I20260927 18:38:53.328418 13326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:53.324135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:44809 (local address 127.11.251.194:42409)
0927 18:38:53.324285 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:53.324293 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:53.324335 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:53.324522 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:53.324523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:53.324526 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:53.324630 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:53.324632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.325667 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.325668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:53.327087 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:53.327090 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.328024 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.328025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:53.328143 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:53.328149 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:38:53.328233 (+ 84us) client_negotiation.cc:240] Negotiation successful
0927 18:38:53.328278 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":104,"thread_start_us":70,"threads_started":1}
I20260927 18:38:53.328869 12986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98190c3088bc48e38c43911fe993b889" candidate_uuid: "fb0ee83254084df2833fbd736002d333" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28528f946190408ab095c1201d592c5f" is_pre_election: true
I20260927 18:38:53.328888 13254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98190c3088bc48e38c43911fe993b889" candidate_uuid: "fb0ee83254084df2833fbd736002d333" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb987313423143cda7324b6e212b32f4" is_pre_election: true
I20260927 18:38:53.329097 13254 raft_consensus.cc:2454] T 98190c3088bc48e38c43911fe993b889 P cb987313423143cda7324b6e212b32f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb0ee83254084df2833fbd736002d333 in term 0.
I20260927 18:38:53.329090 12986 raft_consensus.cc:2454] T 98190c3088bc48e38c43911fe993b889 P 28528f946190408ab095c1201d592c5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb0ee83254084df2833fbd736002d333 in term 0.
I20260927 18:38:53.329433 13056 leader_election.cc:302] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28528f946190408ab095c1201d592c5f, fb0ee83254084df2833fbd736002d333; no voters:
I20260927 18:38:53.329600 13325 raft_consensus.cc:2781] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:38:53.329665 13325 raft_consensus.cc:482] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:38:53.329708 13325 raft_consensus.cc:3037] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:53.330963 13325 raft_consensus.cc:504] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb987313423143cda7324b6e212b32f4" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44809 } } peers { permanent_uuid: "fb0ee83254084df2833fbd736002d333" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 39091 } } peers { permanent_uuid: "28528f946190408ab095c1201d592c5f" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43435 } }
I20260927 18:38:53.331147 13325 leader_election.cc:288] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333 [CANDIDATE]: Term 1 election: Requested vote from peers cb987313423143cda7324b6e212b32f4 (127.11.251.195:44809), 28528f946190408ab095c1201d592c5f (127.11.251.193:43435)
I20260927 18:38:53.331351 13254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98190c3088bc48e38c43911fe993b889" candidate_uuid: "fb0ee83254084df2833fbd736002d333" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb987313423143cda7324b6e212b32f4"
I20260927 18:38:53.331391 12986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98190c3088bc48e38c43911fe993b889" candidate_uuid: "fb0ee83254084df2833fbd736002d333" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28528f946190408ab095c1201d592c5f"
I20260927 18:38:53.331453 13254 raft_consensus.cc:3037] T 98190c3088bc48e38c43911fe993b889 P cb987313423143cda7324b6e212b32f4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:53.331540 12986 raft_consensus.cc:3037] T 98190c3088bc48e38c43911fe993b889 P 28528f946190408ab095c1201d592c5f [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:53.332259 13254 raft_consensus.cc:2454] T 98190c3088bc48e38c43911fe993b889 P cb987313423143cda7324b6e212b32f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb0ee83254084df2833fbd736002d333 in term 1.
I20260927 18:38:53.332506 13055 leader_election.cc:302] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb987313423143cda7324b6e212b32f4, fb0ee83254084df2833fbd736002d333; no voters:
I20260927 18:38:53.332609 12986 raft_consensus.cc:2454] T 98190c3088bc48e38c43911fe993b889 P 28528f946190408ab095c1201d592c5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb0ee83254084df2833fbd736002d333 in term 1.
I20260927 18:38:53.332691 13325 raft_consensus.cc:2781] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:53.332878 13325 raft_consensus.cc:686] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333 [term 1 LEADER]: Becoming Leader. State: Replica: fb0ee83254084df2833fbd736002d333, State: Running, Role: LEADER
I20260927 18:38:53.333014 13325 consensus_queue.cc:237] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb987313423143cda7324b6e212b32f4" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44809 } } peers { permanent_uuid: "fb0ee83254084df2833fbd736002d333" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 39091 } } peers { permanent_uuid: "28528f946190408ab095c1201d592c5f" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43435 } }
I20260927 18:38:53.334075 12841 catalog_manager.cc:5495] T 98190c3088bc48e38c43911fe993b889 P fb0ee83254084df2833fbd736002d333 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb0ee83254084df2833fbd736002d333 (127.11.251.194). New cstate: current_term: 1 leader_uuid: "fb0ee83254084df2833fbd736002d333" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb987313423143cda7324b6e212b32f4" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb0ee83254084df2833fbd736002d333" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 39091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28528f946190408ab095c1201d592c5f" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43435 } health_report { overall_health: UNKNOWN } } }
Sep 27 18:38:53 dist-test-slave-z47j krb5kdc[12796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.193@KRBTEST.COM
I20260927 18:38:53.398885 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:53.390241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59186 (local address 127.11.251.193:43435)
0927 18:38:53.390331 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:53.390333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:53.390430 (+ 97us) server_negotiation.cc:407] Connection header received
0927 18:38:53.390488 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:53.390489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:53.390502 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:53.390568 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:53.391366 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.391933 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.392880 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.393142 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.393232 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:53.395831 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:53.395844 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:53.395844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:53.395852 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:53.397786 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:53.398138 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:53.398141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:53.398142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:53.398212 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:53.398400 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:53.398401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:53.398401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:53.398555 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:53.398686 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:53.398722 (+ 36us) server_negotiation.cc:299] Negotiation successful
0927 18:38:53.398805 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Sep 27 18:38:53 dist-test-slave-z47j krb5kdc[12796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534333, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 18:38:53 dist-test-slave-z47j krb5kdc[12796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534333, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.251.193@KRBTEST.COM
I20260927 18:38:53.424602 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:53.416260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59190 (local address 127.11.251.193:43435)
0927 18:38:53.416360 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:53.416363 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:53.416467 (+ 104us) server_negotiation.cc:407] Connection header received
0927 18:38:53.416601 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:53.416602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:53.416616 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:53.416724 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:53.417561 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.418256 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.419062 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.419287 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.419376 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:53.421773 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:53.421775 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:53.421775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:53.421782 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:53.423257 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:53.423648 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:53.423649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:53.423650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:53.423731 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:53.423965 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:53.423966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:53.423967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:53.424089 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:53.424202 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:53.424409 (+ 207us) server_negotiation.cc:299] Negotiation successful
0927 18:38:53.424491 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Sep 27 18:38:53 dist-test-slave-z47j krb5kdc[12796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 18:38:53 dist-test-slave-z47j krb5kdc[12796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.193@KRBTEST.COM
I20260927 18:38:53.451824 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:53.444020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59192 (local address 127.11.251.193:43435)
0927 18:38:53.444103 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:53.444106 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:53.444252 (+ 146us) server_negotiation.cc:407] Connection header received
0927 18:38:53.444345 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:53.444346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:53.444358 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:53.444416 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:53.445002 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.445495 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.446400 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:53.446591 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:53.446669 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:53.448983 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:53.448985 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:53.448985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:53.448992 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:53.450549 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:53.450947 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:53.450949 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:53.450949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:53.451028 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:53.451247 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:53.451248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:53.451249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:53.451359 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:53.451478 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:53.451627 (+ 149us) server_negotiation.cc:299] Negotiation successful
0927 18:38:53.451709 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260927 18:38:53.452450 12946 tablet_service.cc:2908] Rejecting scan request for tablet 98190c3088bc48e38c43911fe993b889: Uninitialized: clean time has not yet been initialized
I20260927 18:38:53.453540 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 12903
I20260927 18:38:53.459980 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 13037
I20260927 18:38:53.467635 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 13171
I20260927 18:38:53.473214 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 12809
2026-09-27T18:38:53Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2371 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:38:53 dist-test-slave-z47j krb5kdc[13341](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:38:53 dist-test-slave-z47j krb5kdc[13341](info): set up 2 sockets
Sep 27 18:38:53 dist-test-slave-z47j krb5kdc[13341](info): commencing operation
krb5kdc: starting...
W20260927 18:38:54.517983 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:38:54 dist-test-slave-z47j krb5kdc[13341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:38:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:38:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:38:54.653524 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:45193
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:45193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:54.744083 13354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:54.744320 13354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:54.744351 13354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:54.745689 13354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:38:54.745731 13354 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 18:38:54.745745 13354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:54.745759 13354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:54.745771 13354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:38:54.745817 13354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:38:54.747349 13354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:45193
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:45193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:38:54.747624 13354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:54.747888 13354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:54.753489 13362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:54.753539 13359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:54.753513 13360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:54.753610 13354 server_base.cc:1034] running on GCE node
I20260927 18:38:54.754220 13354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:54.754572 13354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:54.755827 13354 hybrid_clock.cc:648] HybridClock initialized: now 1790534334755798 us; error 67 us; skew 500 ppm
Sep 27 18:38:54 dist-test-slave-z47j krb5kdc[13341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534334, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:54.758919 13354 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:38:54.759503 13354 webserver.cc:460] Webserver started at http://127.11.251.254:39647/ using document root <none> and password file <none>
I20260927 18:38:54.759789 13354 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:54.759861 13354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:54.760023 13354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:54.761207 13354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "77cec5a02f4c4e1d837807a85391cf54"
format_stamp: "Formatted at 2026-09-27 18:38:54 on dist-test-slave-z47j"
server_key: "9b9cf8319344c508f2732061c6f42cdb"
server_key_iv: "000b55d65659efdfd8965b1527f5a165"
server_key_version: "encryptionkey@0"
I20260927 18:38:54.761668 13354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "77cec5a02f4c4e1d837807a85391cf54"
format_stamp: "Formatted at 2026-09-27 18:38:54 on dist-test-slave-z47j"
server_key: "9b9cf8319344c508f2732061c6f42cdb"
server_key_iv: "000b55d65659efdfd8965b1527f5a165"
server_key_version: "encryptionkey@0"
I20260927 18:38:54.763173 13354 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:54.764207 13369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:54.764505 13354 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:54.764642 13354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "77cec5a02f4c4e1d837807a85391cf54"
format_stamp: "Formatted at 2026-09-27 18:38:54 on dist-test-slave-z47j"
server_key: "9b9cf8319344c508f2732061c6f42cdb"
server_key_iv: "000b55d65659efdfd8965b1527f5a165"
server_key_version: "encryptionkey@0"
I20260927 18:38:54.764741 13354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:54.796378 13354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:54.798900 13354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:54.799098 13354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:54.805240 13354 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:45193
I20260927 18:38:54.805327 13431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:45193 every 8 connection(s)
I20260927 18:38:54.805806 13354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:38:54.806660 13432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:54.809041 13432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54: Bootstrap starting.
I20260927 18:38:54.809655 13432 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:54.809705 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 13354
I20260927 18:38:54.809808 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:38:54.809986 13432 log.cc:824] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:54.810014 12271 external_mini_cluster.cc:1428] Setting key b1b6d21bb96eef22d8590a4becde06f1
I20260927 18:38:54.810916 13432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54: No bootstrap required, opened a new log
I20260927 18:38:54.813025 13432 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77cec5a02f4c4e1d837807a85391cf54" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 45193 } }
I20260927 18:38:54.813207 13432 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:54.813243 13432 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77cec5a02f4c4e1d837807a85391cf54, State: Initialized, Role: FOLLOWER
I20260927 18:38:54.813396 13432 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77cec5a02f4c4e1d837807a85391cf54" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 45193 } }
I20260927 18:38:54.813485 13432 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:38:54.813535 13432 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:38:54.813614 13432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:54.814270 13432 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77cec5a02f4c4e1d837807a85391cf54" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 45193 } }
I20260927 18:38:54.814385 13432 leader_election.cc:302] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77cec5a02f4c4e1d837807a85391cf54; no voters:
I20260927 18:38:54.814565 13432 leader_election.cc:288] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:38:54.814642 13437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:54.814836 13432 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:38:54.814924 13437 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54 [term 1 LEADER]: Becoming Leader. State: Replica: 77cec5a02f4c4e1d837807a85391cf54, State: Running, Role: LEADER
I20260927 18:38:54.815057 13437 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77cec5a02f4c4e1d837807a85391cf54" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 45193 } }
I20260927 18:38:54.815518 13437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77cec5a02f4c4e1d837807a85391cf54. Latest consensus state: current_term: 1 leader_uuid: "77cec5a02f4c4e1d837807a85391cf54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77cec5a02f4c4e1d837807a85391cf54" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 45193 } } }
I20260927 18:38:54.815531 13438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77cec5a02f4c4e1d837807a85391cf54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77cec5a02f4c4e1d837807a85391cf54" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 45193 } } }
I20260927 18:38:54.815723 13437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:54.815732 13438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:54.816009 13446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:38:54.816648 13446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:38:54.818684 13446 catalog_manager.cc:1347] Generated new cluster ID: 04d214ae90254e74a57b445f525141bb
I20260927 18:38:54.818743 13446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:38:54 dist-test-slave-z47j krb5kdc[13341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:38:54.828027 13436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:54.810631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39318 (local address 127.11.251.254:45193)
0927 18:38:54.810868 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:54.810878 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:54.810916 (+ 38us) server_negotiation.cc:407] Connection header received
0927 18:38:54.811440 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:54.811449 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:54.811518 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:54.811691 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:54.812368 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:54.814221 (+ 1853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:54.815451 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:54.816154 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:54.816270 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:54.823971 (+ 7701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:54.823978 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:54.823978 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:54.823993 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:54.826150 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:54.826623 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:54.826626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:54.826627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:54.826738 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:54.826981 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:54.826981 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:54.826982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:54.827306 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:54.827467 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:54.827609 (+ 142us) server_negotiation.cc:299] Negotiation successful
0927 18:38:54.827768 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":107,"threads_started":1}
I20260927 18:38:54.835170 13446 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:38:54.836002 13446 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:38:54.842033 13446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54: Generated new TSK 0
I20260927 18:38:54.842255 13446 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.251.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
I20260927 18:38:54.872874 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:45193
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:54.962852 13458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:54.963084 13458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:54.963109 13458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:54.964277 13458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:54.964313 13458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:54.964334 13458 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:38:54.964352 13458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:38:54.965812 13458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.251.254:45193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:38:54.966027 13458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:54.966250 13458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:38:54.971918 13458 server_base.cc:1034] running on GCE node
W20260927 18:38:54.971882 13464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:54.971882 13463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:54.971866 13466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:54.972445 13458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:54.972759 13458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:54.973928 13458 hybrid_clock.cc:648] HybridClock initialized: now 1790534334973925 us; error 39 us; skew 500 ppm
Sep 27 18:38:54 dist-test-slave-z47j krb5kdc[13341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534334, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:54.976691 13458 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:38:54.977177 13458 webserver.cc:460] Webserver started at http://127.11.251.193:36261/ using document root <none> and password file <none>
I20260927 18:38:54.977449 13458 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:54.977517 13458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:54.977686 13458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:54.978856 13458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "ce4909f4936245de9d3ae7cdf436a23d"
format_stamp: "Formatted at 2026-09-27 18:38:54 on dist-test-slave-z47j"
server_key: "9650dc418f316cba2abc141dab6f0c88"
server_key_iv: "d08fe71bc7af8dd48a54a4776831ad91"
server_key_version: "encryptionkey@0"
I20260927 18:38:54.979223 13458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "ce4909f4936245de9d3ae7cdf436a23d"
format_stamp: "Formatted at 2026-09-27 18:38:54 on dist-test-slave-z47j"
server_key: "9650dc418f316cba2abc141dab6f0c88"
server_key_iv: "d08fe71bc7af8dd48a54a4776831ad91"
server_key_version: "encryptionkey@0"
I20260927 18:38:54.980692 13458 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:38:54.981755 13473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:54.982100 13458 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:54.982229 13458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "ce4909f4936245de9d3ae7cdf436a23d"
format_stamp: "Formatted at 2026-09-27 18:38:54 on dist-test-slave-z47j"
server_key: "9650dc418f316cba2abc141dab6f0c88"
server_key_iv: "d08fe71bc7af8dd48a54a4776831ad91"
server_key_version: "encryptionkey@0"
I20260927 18:38:54.982308 13458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:55.000162 13458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:55.003330 13458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:55.003499 13458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:55.004251 13458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:55.004361 13458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:55.004417 13458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:55.004495 13458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:55.012895 13458 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:38593
I20260927 18:38:55.012988 13588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:38593 every 8 connection(s)
Sep 27 18:38:55 dist-test-slave-z47j krb5kdc[13341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534334, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:38:55.013386 13458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:38:55.018049 13436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.005450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:58669 (local address 127.11.251.254:45193)
0927 18:38:55.005501 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:55.005504 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:55.005517 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:38:55.006193 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:55.006195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:55.006210 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:55.006514 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:55.007471 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.008179 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.010008 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.010266 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.010347 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.013938 (+ 3591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:55.013940 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:55.013941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:55.013950 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:55.015969 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:55.016700 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:55.016702 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:55.016702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:55.016767 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:55.017167 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:55.017168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:55.017169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:55.017356 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:55.017437 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:55.017877 (+ 440us) server_negotiation.cc:299] Negotiation successful
0927 18:38:55.017965 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:38:55.018356 13482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.004720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:45193 (local address 127.11.251.193:58669)
0927 18:38:55.005205 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:55.005262 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:55.006030 (+ 768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:55.006648 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:55.006654 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:55.007057 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:55.007354 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.007360 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.008311 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.008313 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.009915 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.009920 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.010368 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.010369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.010547 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.011760 (+ 1213us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:55.011770 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:55.013821 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:55.016175 (+ 2354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:55.016183 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:55.016185 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:55.016591 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:55.016842 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:55.016843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:55.016843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:55.017004 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:55.017443 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:55.017451 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:55.017697 (+ 246us) client_negotiation.cc:769] Sending connection context
0927 18:38:55.017836 (+ 139us) client_negotiation.cc:240] Negotiation successful
0927 18:38:55.018048 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":378,"threads_started":1}
I20260927 18:38:55.018729 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 13458
I20260927 18:38:55.018882 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:38:55.019021 13589 heartbeater.cc:347] Connected to a master server at 127.11.251.254:45193
I20260927 18:38:55.019124 13589 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:55.019124 12271 external_mini_cluster.cc:1428] Setting key bc7af66ba51b469000963e37814526a2
I20260927 18:38:55.019364 13589 heartbeater.cc:511] Master 127.11.251.254:45193 requested a full tablet report, sending...
I20260927 18:38:55.020161 13386 ts_manager.cc:194] Registered new tserver with Master: ce4909f4936245de9d3ae7cdf436a23d (127.11.251.193:38593)
I20260927 18:38:55.021102 13386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:58669
I20260927 18:38:55.028013 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.020155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:45193 (local address 127.11.251.193:36735)
0927 18:38:55.020298 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:55.020308 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:55.020383 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:55.020516 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:55.020517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:55.020710 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:55.020762 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.020764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.021419 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.021420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.022340 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.022343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.022885 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.022886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.022965 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.023483 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:55.023488 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:55.023811 (+ 323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:55.026821 (+ 3010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:55.026822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:55.026823 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:55.027119 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:55.027332 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:55.027333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:55.027333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:55.027398 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:55.027640 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:55.027641 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:55.027687 (+ 46us) client_negotiation.cc:769] Sending connection context
0927 18:38:55.027783 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 18:38:55.027909 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260927 18:38:55.028038 13436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.020204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:36735 (local address 127.11.251.254:45193)
0927 18:38:55.020243 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:55.020245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:55.020368 (+ 123us) server_negotiation.cc:407] Connection header received
0927 18:38:55.020441 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:55.020442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:55.020452 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:55.020506 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:55.020833 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.021325 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.022458 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.022804 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.022885 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.023921 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:55.023922 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:55.023923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:55.023930 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:55.026708 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:55.027195 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:55.027196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:55.027197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:55.027258 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:55.027459 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:55.027460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:55.027461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:55.027576 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:55.027648 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:55.027871 (+ 223us) server_negotiation.cc:299] Negotiation successful
0927 18:38:55.027955 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for kudu/127.11.251.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
I20260927 18:38:55.052443 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:45193
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:55.138705 13598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:55.138932 13598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:55.138968 13598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:55.140096 13598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:55.140156 13598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:55.140189 13598 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:38:55.140226 13598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:38:55.141672 13598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.251.254:45193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:38:55.141912 13598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:55.142131 13598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:55.147264 13606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:55.147325 13604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:55.147621 13603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:55.147935 13598 server_base.cc:1034] running on GCE node
I20260927 18:38:55.148247 13598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:55.148547 13598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:55.149704 13598 hybrid_clock.cc:648] HybridClock initialized: now 1790534335149688 us; error 43 us; skew 500 ppm
Sep 27 18:38:55 dist-test-slave-z47j krb5kdc[13341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534335, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:55.152592 13598 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:38:55.153072 13598 webserver.cc:460] Webserver started at http://127.11.251.194:34505/ using document root <none> and password file <none>
I20260927 18:38:55.153286 13598 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:55.153332 13598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:55.153430 13598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:55.154659 13598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "f55e2417f3ad47a7997d0e9162396c9b"
format_stamp: "Formatted at 2026-09-27 18:38:55 on dist-test-slave-z47j"
server_key: "729739d489f7d8863eb9d364bbc0728d"
server_key_iv: "36bdf7ccbc8ac95986257a6a00c0b4fd"
server_key_version: "encryptionkey@0"
I20260927 18:38:55.154989 13598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "f55e2417f3ad47a7997d0e9162396c9b"
format_stamp: "Formatted at 2026-09-27 18:38:55 on dist-test-slave-z47j"
server_key: "729739d489f7d8863eb9d364bbc0728d"
server_key_iv: "36bdf7ccbc8ac95986257a6a00c0b4fd"
server_key_version: "encryptionkey@0"
I20260927 18:38:55.156378 13598 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:38:55.157195 13613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:55.157446 13598 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:55.157543 13598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "f55e2417f3ad47a7997d0e9162396c9b"
format_stamp: "Formatted at 2026-09-27 18:38:55 on dist-test-slave-z47j"
server_key: "729739d489f7d8863eb9d364bbc0728d"
server_key_iv: "36bdf7ccbc8ac95986257a6a00c0b4fd"
server_key_version: "encryptionkey@0"
I20260927 18:38:55.157648 13598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:55.168947 13598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:55.171305 13598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:55.171487 13598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:55.172097 13598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:55.172137 13598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:55.172168 13598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:55.172214 13598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:38:55 dist-test-slave-z47j krb5kdc[13341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534335, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:38:55.181154 13598 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:36383
I20260927 18:38:55.181558 13598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:38:55.181641 13728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:36383 every 8 connection(s)
I20260927 18:38:55.183708 13436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.172744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:38233 (local address 127.11.251.254:45193)
0927 18:38:55.172799 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:55.172801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:55.173117 (+ 316us) server_negotiation.cc:407] Connection header received
0927 18:38:55.173778 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:55.173779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:55.173791 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:55.173848 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:55.174840 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.175757 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.177452 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.177841 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.177918 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.180674 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:55.180675 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:55.180676 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:55.180684 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:55.182327 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:55.182750 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:55.182752 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:55.182752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:55.182811 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:55.183068 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:55.183069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:55.183070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:55.183193 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:55.183264 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:55.183531 (+ 267us) server_negotiation.cc:299] Negotiation successful
0927 18:38:55.183614 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 18:38:55.183910 13622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.172771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:45193 (local address 127.11.251.194:38233)
0927 18:38:55.172981 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:55.173023 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:55.173660 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:55.173936 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:55.173941 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:55.174361 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:55.174667 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.174673 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.175892 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.175893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.177299 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.177302 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.177968 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.177968 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.178120 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.178816 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:55.178828 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:55.180576 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:55.182429 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:55.182435 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:55.182438 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:55.182662 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:55.182874 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:55.182875 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:55.182875 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:55.182995 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:55.183256 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:55.183263 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:55.183400 (+ 137us) client_negotiation.cc:769] Sending connection context
0927 18:38:55.183510 (+ 110us) client_negotiation.cc:240] Negotiation successful
0927 18:38:55.183671 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":108,"threads_started":1}
I20260927 18:38:55.184490 13729 heartbeater.cc:347] Connected to a master server at 127.11.251.254:45193
I20260927 18:38:55.184580 13729 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:55.184787 13729 heartbeater.cc:511] Master 127.11.251.254:45193 requested a full tablet report, sending...
I20260927 18:38:55.185325 13385 ts_manager.cc:194] Registered new tserver with Master: f55e2417f3ad47a7997d0e9162396c9b (127.11.251.194:36383)
I20260927 18:38:55.185892 13385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:38233
I20260927 18:38:55.188923 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 13598
I20260927 18:38:55.189014 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:38:55.189213 12271 external_mini_cluster.cc:1428] Setting key 58bd13fea3ddf2ac1493f94e91ea58a7
I20260927 18:38:55.191776 13436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.185678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:51229 (local address 127.11.251.254:45193)
0927 18:38:55.185764 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:55.185766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:55.185827 (+ 61us) server_negotiation.cc:407] Connection header received
0927 18:38:55.185888 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:55.185889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:55.185899 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:55.185977 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:55.186366 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.187043 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.187744 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.187920 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.187970 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.188883 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:55.188885 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:55.188885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:55.188891 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:55.190499 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:55.190901 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:55.190901 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:55.190902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:55.190950 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:55.191158 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:55.191158 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:55.191158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:55.191339 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:55.191426 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:55.191594 (+ 168us) server_negotiation.cc:299] Negotiation successful
0927 18:38:55.191680 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:38:55.191818 13735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.185652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:45193 (local address 127.11.251.194:51229)
0927 18:38:55.185765 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:55.185775 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:55.185838 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:55.185982 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:55.185983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:55.186190 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:55.186283 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.186285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.187130 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.187130 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.187671 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.187673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.187986 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.187986 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.188043 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.188476 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:55.188481 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:55.188791 (+ 310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:55.190583 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:55.190584 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:55.190584 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:55.190819 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:55.191020 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:55.191021 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:55.191021 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:55.191092 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:55.191422 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:55.191423 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:55.191483 (+ 60us) client_negotiation.cc:769] Sending connection context
0927 18:38:55.191569 (+ 86us) client_negotiation.cc:240] Negotiation successful
0927 18:38:55.191695 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":67,"threads_started":1}
WARNING: no policy specified for kudu/127.11.251.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
I20260927 18:38:55.216861 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:45193
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:55.297693 13738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:55.297933 13738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:55.297977 13738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:55.299166 13738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:55.299233 13738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:55.299269 13738 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:38:55.299314 13738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:38:55.300860 13738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.251.254:45193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:38:55.301119 13738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:55.301340 13738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:55.306442 13746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:55.306484 13743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:55.306480 13744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:55.306727 13738 server_base.cc:1034] running on GCE node
I20260927 18:38:55.306946 13738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:55.307211 13738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:55.308383 13738 hybrid_clock.cc:648] HybridClock initialized: now 1790534335308364 us; error 33 us; skew 500 ppm
Sep 27 18:38:55 dist-test-slave-z47j krb5kdc[13341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534335, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:55.310787 13738 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:38:55.311266 13738 webserver.cc:460] Webserver started at http://127.11.251.195:46727/ using document root <none> and password file <none>
I20260927 18:38:55.311492 13738 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:55.311548 13738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:55.311707 13738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:55.312729 13738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "908900369fd744718570aac5645c6bbe"
format_stamp: "Formatted at 2026-09-27 18:38:55 on dist-test-slave-z47j"
server_key: "63f978d2fdd665ebffda036aa8520826"
server_key_iv: "74bdced6f710347c46d6e90f54dd23d1"
server_key_version: "encryptionkey@0"
I20260927 18:38:55.313079 13738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "908900369fd744718570aac5645c6bbe"
format_stamp: "Formatted at 2026-09-27 18:38:55 on dist-test-slave-z47j"
server_key: "63f978d2fdd665ebffda036aa8520826"
server_key_iv: "74bdced6f710347c46d6e90f54dd23d1"
server_key_version: "encryptionkey@0"
I20260927 18:38:55.314606 13738 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:38:55.315500 13753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:55.315716 13738 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:38:55.315826 13738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "908900369fd744718570aac5645c6bbe"
format_stamp: "Formatted at 2026-09-27 18:38:55 on dist-test-slave-z47j"
server_key: "63f978d2fdd665ebffda036aa8520826"
server_key_iv: "74bdced6f710347c46d6e90f54dd23d1"
server_key_version: "encryptionkey@0"
I20260927 18:38:55.315909 13738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:55.340142 13738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:55.342589 13738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:55.342770 13738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:55.343401 13738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:55.343483 13738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:55.343529 13738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:55.343554 13738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:38:55 dist-test-slave-z47j krb5kdc[13341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534335, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:38:55.352278 13738 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:45447
I20260927 18:38:55.352442 13868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:45447 every 8 connection(s)
I20260927 18:38:55.352838 13738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:38:55.355295 13436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.343847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:36779 (local address 127.11.251.254:45193)
0927 18:38:55.343927 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:55.343930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:55.344137 (+ 207us) server_negotiation.cc:407] Connection header received
0927 18:38:55.344675 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:55.344677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:55.344689 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:55.344745 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:55.345602 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.346348 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.347885 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.348290 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.348414 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.352233 (+ 3819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:55.352235 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:55.352235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:55.352245 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:55.353922 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:55.354424 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:55.354424 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:55.354425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:55.354474 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:55.354745 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:55.354745 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:55.354745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:55.354850 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:55.354910 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:55.355165 (+ 255us) server_negotiation.cc:299] Negotiation successful
0927 18:38:55.355230 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:38:55.355594 13762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.343818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:45193 (local address 127.11.251.195:36779)
0927 18:38:55.344012 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:55.344052 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:55.344565 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:55.344819 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:55.344822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:55.345179 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:55.345449 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.345455 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.346429 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.346430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.347745 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.347747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.348435 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.348436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.348577 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.349204 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:55.349215 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:55.352092 (+ 2877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:55.354017 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:55.354022 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:55.354025 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:55.354342 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:55.354557 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:55.354558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:55.354559 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:55.354679 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:55.354900 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:55.354904 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:55.355037 (+ 133us) client_negotiation.cc:769] Sending connection context
0927 18:38:55.355177 (+ 140us) client_negotiation.cc:240] Negotiation successful
0927 18:38:55.355331 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":122,"threads_started":1}
I20260927 18:38:55.356199 13869 heartbeater.cc:347] Connected to a master server at 127.11.251.254:45193
I20260927 18:38:55.356297 13869 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:55.356523 13869 heartbeater.cc:511] Master 127.11.251.254:45193 requested a full tablet report, sending...
I20260927 18:38:55.356992 13386 ts_manager.cc:194] Registered new tserver with Master: 908900369fd744718570aac5645c6bbe (127.11.251.195:45447)
I20260927 18:38:55.357551 13386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:36779
I20260927 18:38:55.362689 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 13738
I20260927 18:38:55.362800 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:38:55.362978 12271 external_mini_cluster.cc:1428] Setting key 49d352f8d7fc4fc1d5f029408278220c
I20260927 18:38:55.363935 13875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.357431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:45193 (local address 127.11.251.195:32973)
0927 18:38:55.357599 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:55.357610 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:55.357669 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:55.357879 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:55.357880 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:55.358106 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:55.358182 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.358185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.358965 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.358966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.359495 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.359497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.359866 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.359867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.359932 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.360411 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:55.360416 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:55.360801 (+ 385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:55.362688 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:55.362690 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:55.362690 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:55.362936 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:55.363203 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:55.363203 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:55.363204 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:55.363289 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:55.363599 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:55.363599 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:55.363664 (+ 65us) client_negotiation.cc:769] Sending connection context
0927 18:38:55.363751 (+ 87us) client_negotiation.cc:240] Negotiation successful
0927 18:38:55.363855 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":160,"threads_started":1}
I20260927 18:38:55.363972 13436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.357501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:32973 (local address 127.11.251.254:45193)
0927 18:38:55.357555 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:55.357558 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:55.357654 (+ 96us) server_negotiation.cc:407] Connection header received
0927 18:38:55.357768 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:55.357769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:55.357783 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:55.357869 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:55.358334 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.358840 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.359581 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.359766 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.359828 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.360882 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:55.360883 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:55.360883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:55.360890 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:55.362587 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:55.363045 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:55.363046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:55.363047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:55.363116 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:55.363374 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:55.363375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:55.363375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:55.363521 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:55.363593 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:55.363789 (+ 196us) server_negotiation.cc:299] Negotiation successful
0927 18:38:55.363871 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 18:38:55.364209 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 18:38:55 dist-test-slave-z47j krb5kdc[13341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534335, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 18:38:55 dist-test-slave-z47j krb5kdc[13341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534335, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:38:55.389845 13436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.382333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39326 (local address 127.11.251.254:45193)
0927 18:38:55.382411 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:55.382414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:55.382424 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:38:55.382482 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:55.382483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:55.382494 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:55.382556 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:55.383157 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.383719 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.384580 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.384776 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.384833 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.387320 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:55.387321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:55.387321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:55.387328 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:55.388711 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:55.389051 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:55.389051 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:55.389052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:55.389104 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:55.389275 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:55.389276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:55.389276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:55.389379 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:55.389463 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:55.389713 (+ 250us) server_negotiation.cc:299] Negotiation successful
0927 18:38:55.389775 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 18:38:55.391743 13386 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:38:55.393406 13386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:38:55.400596 13888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.396770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:45447 (local address 127.0.0.1:53874)
0927 18:38:55.396913 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:55.396923 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:55.396966 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:55.397454 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:55.397457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:55.397459 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:55.397526 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.397528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.398503 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.398504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.399556 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.399559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.400317 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.400319 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.400412 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.400430 (+ 18us) client_negotiation.cc:769] Sending connection context
0927 18:38:55.400489 (+ 59us) client_negotiation.cc:240] Negotiation successful
0927 18:38:55.400539 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":104,"threads_started":1}
I20260927 18:38:55.401018 13890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.397067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53874 (local address 127.11.251.195:45447)
0927 18:38:55.397292 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:55.397297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:55.397305 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:38:55.397335 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:55.397337 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:55.397349 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:55.397441 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:55.397624 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.398409 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.399648 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.400220 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.400811 (+ 591us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.400874 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:55.400898 (+ 24us) server_negotiation.cc:299] Negotiation successful
0927 18:38:55.400944 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":90,"threads_started":1}
I20260927 18:38:55.401544 13886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.396539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:36383 (local address 127.0.0.1:51188)
0927 18:38:55.396658 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:55.396700 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:55.396765 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:55.397487 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:55.397489 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:55.397492 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:55.397605 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.397609 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.399049 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.399051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.400344 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.400346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.401299 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.401300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.401356 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.401358 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:38:55.401391 (+ 33us) client_negotiation.cc:240] Negotiation successful
0927 18:38:55.401445 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":61,"thread_start_us":69,"threads_started":1}
I20260927 18:38:55.401691 13891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.397038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41336 (local address 127.11.251.193:38593)
0927 18:38:55.397660 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:55.397665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:55.397675 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:38:55.397714 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:55.397716 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:55.397744 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:55.397836 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:55.398009 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.399058 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.400313 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.401090 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.401170 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.401242 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:55.401493 (+ 251us) server_negotiation.cc:299] Negotiation successful
0927 18:38:55.401535 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":130,"threads_started":1}
I20260927 18:38:55.401796 13803 tablet_service.cc:1467] Processing CreateTablet for tablet 106ea2eb57224cfbbbe957ef7e7fca2b (DEFAULT_TABLE table=test-table [id=4d3253fb05c249fa8091a506fb1ce21c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:55.402060 13889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.396786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51188 (local address 127.11.251.194:36383)
0927 18:38:55.397300 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:55.397306 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:55.397318 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:38:55.397365 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:55.397366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:55.397380 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:55.397476 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:55.397696 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.398942 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.400430 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.401201 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.401858 (+ 657us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.401899 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:55.401919 (+ 20us) server_negotiation.cc:299] Negotiation successful
0927 18:38:55.401952 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":123,"threads_started":1}
I20260927 18:38:55.402218 13887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.396542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:38593 (local address 127.0.0.1:41336)
0927 18:38:55.396701 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:55.396714 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:55.396778 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:55.397861 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:55.397862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:55.397865 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:55.397908 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.397909 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.399150 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.399151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.400219 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.400222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.401284 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.401285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.401354 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.401361 (+ 7us) client_negotiation.cc:769] Sending connection context
0927 18:38:55.401409 (+ 48us) client_negotiation.cc:240] Negotiation successful
0927 18:38:55.401446 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":3,"thread_start_us":115,"threads_started":1}
I20260927 18:38:55.402236 13803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 106ea2eb57224cfbbbe957ef7e7fca2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:55.402743 13663 tablet_service.cc:1467] Processing CreateTablet for tablet 106ea2eb57224cfbbbe957ef7e7fca2b (DEFAULT_TABLE table=test-table [id=4d3253fb05c249fa8091a506fb1ce21c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:55.402915 13523 tablet_service.cc:1467] Processing CreateTablet for tablet 106ea2eb57224cfbbbe957ef7e7fca2b (DEFAULT_TABLE table=test-table [id=4d3253fb05c249fa8091a506fb1ce21c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:55.403136 13663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 106ea2eb57224cfbbbe957ef7e7fca2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:55.403272 13523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 106ea2eb57224cfbbbe957ef7e7fca2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:55.405723 13892 tablet_bootstrap.cc:492] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe: Bootstrap starting.
I20260927 18:38:55.407021 13892 tablet_bootstrap.cc:654] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:55.407039 13894 tablet_bootstrap.cc:492] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d: Bootstrap starting.
I20260927 18:38:55.407366 13892 log.cc:824] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:55.407635 13894 tablet_bootstrap.cc:654] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:55.407940 13894 log.cc:824] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:55.406822 13893 tablet_bootstrap.cc:492] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b: Bootstrap starting.
I20260927 18:38:55.408784 13894 tablet_bootstrap.cc:492] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d: No bootstrap required, opened a new log
I20260927 18:38:55.408869 13893 tablet_bootstrap.cc:654] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:55.408871 13894 ts_tablet_manager.cc:1397] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:55.409207 13893 log.cc:824] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:55.410044 13893 tablet_bootstrap.cc:492] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b: No bootstrap required, opened a new log
I20260927 18:38:55.410044 13892 tablet_bootstrap.cc:492] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe: No bootstrap required, opened a new log
I20260927 18:38:55.410130 13892 ts_tablet_manager.cc:1397] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260927 18:38:55.410130 13893 ts_tablet_manager.cc:1397] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 18:38:55.410737 13894 raft_consensus.cc:348] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:55.410888 13894 raft_consensus.cc:374] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:55.410933 13894 raft_consensus.cc:729] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce4909f4936245de9d3ae7cdf436a23d, State: Initialized, Role: FOLLOWER
I20260927 18:38:55.411077 13894 consensus_queue.cc:260] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:55.411350 13589 heartbeater.cc:503] Master 127.11.251.254:45193 was elected leader, sending a full tablet report...
I20260927 18:38:55.411339 13894 ts_tablet_manager.cc:1428] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 18:38:55.412369 13892 raft_consensus.cc:348] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:55.412402 13893 raft_consensus.cc:348] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:55.412546 13893 raft_consensus.cc:374] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:55.412546 13892 raft_consensus.cc:374] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:55.412590 13892 raft_consensus.cc:729] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 908900369fd744718570aac5645c6bbe, State: Initialized, Role: FOLLOWER
I20260927 18:38:55.412590 13893 raft_consensus.cc:729] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f55e2417f3ad47a7997d0e9162396c9b, State: Initialized, Role: FOLLOWER
I20260927 18:38:55.412711 13893 consensus_queue.cc:260] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:55.412711 13892 consensus_queue.cc:260] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:55.412992 13893 ts_tablet_manager.cc:1428] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:38:55.412992 13892 ts_tablet_manager.cc:1428] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:38:55.413040 13729 heartbeater.cc:503] Master 127.11.251.254:45193 was elected leader, sending a full tablet report...
I20260927 18:38:55.413133 13869 heartbeater.cc:503] Master 127.11.251.254:45193 was elected leader, sending a full tablet report...
W20260927 18:38:55.432366 13730 tablet.cc:2369] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:38:55.514755 13590 tablet.cc:2369] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:38:55.603577 13870 tablet.cc:2369] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:38:55.646392 13899 raft_consensus.cc:482] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:38:55.646581 13899 raft_consensus.cc:504] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:55.647056 13899 leader_election.cc:288] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f55e2417f3ad47a7997d0e9162396c9b (127.11.251.194:36383), ce4909f4936245de9d3ae7cdf436a23d (127.11.251.193:38593)
I20260927 18:38:55.650467 13891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.647474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:49733 (local address 127.11.251.193:38593)
0927 18:38:55.647565 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:55.647567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:55.647744 (+ 177us) server_negotiation.cc:407] Connection header received
0927 18:38:55.647857 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:55.647858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:55.647867 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:55.647922 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:55.648128 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.648665 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.649693 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.650070 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.650133 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.650163 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:55.650333 (+ 170us) server_negotiation.cc:299] Negotiation successful
0927 18:38:55.650387 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 18:38:55.650467 13901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.647401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:38593 (local address 127.11.251.195:49733)
0927 18:38:55.647639 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:55.647650 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:55.647771 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:55.647948 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:55.647948 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:55.647951 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:55.648046 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.648049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.648757 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.648758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.649533 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.649535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.650159 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.650160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.650252 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.650255 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:38:55.650302 (+ 47us) client_negotiation.cc:240] Negotiation successful
0927 18:38:55.650339 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":3,"thread_start_us":166,"threads_started":1}
I20260927 18:38:55.650535 13889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.647334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:57001 (local address 127.11.251.194:36383)
0927 18:38:55.647407 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:55.647409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:55.647419 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:38:55.647452 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:55.647452 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:55.647464 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:55.647549 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:55.647816 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.648516 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.649704 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.650082 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.650189 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.650201 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:55.650400 (+ 199us) server_negotiation.cc:299] Negotiation successful
0927 18:38:55.650445 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 18:38:55.650818 13762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.647241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:36383 (local address 127.11.251.195:57001)
0927 18:38:55.647312 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:55.647325 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:55.647374 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:55.647606 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:55.647607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:55.647608 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:55.647675 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.647677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.648613 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.648614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.649530 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.649533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.650195 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.650196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.650307 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.650312 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:38:55.650377 (+ 65us) client_negotiation.cc:240] Negotiation successful
0927 18:38:55.650427 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":29}
I20260927 18:38:55.651016 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106ea2eb57224cfbbbe957ef7e7fca2b" candidate_uuid: "908900369fd744718570aac5645c6bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce4909f4936245de9d3ae7cdf436a23d" is_pre_election: true
I20260927 18:38:55.651245 13543 raft_consensus.cc:2454] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 908900369fd744718570aac5645c6bbe in term 0.
I20260927 18:38:55.651248 13683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106ea2eb57224cfbbbe957ef7e7fca2b" candidate_uuid: "908900369fd744718570aac5645c6bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f55e2417f3ad47a7997d0e9162396c9b" is_pre_election: true
I20260927 18:38:55.651448 13683 raft_consensus.cc:2454] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 908900369fd744718570aac5645c6bbe in term 0.
I20260927 18:38:55.651520 13754 leader_election.cc:302] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 908900369fd744718570aac5645c6bbe, ce4909f4936245de9d3ae7cdf436a23d; no voters:
I20260927 18:38:55.651683 13899 raft_consensus.cc:2781] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:38:55.651754 13899 raft_consensus.cc:482] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:38:55.651788 13899 raft_consensus.cc:3037] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:55.652979 13899 raft_consensus.cc:504] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:55.653194 13899 leader_election.cc:288] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe [CANDIDATE]: Term 1 election: Requested vote from peers f55e2417f3ad47a7997d0e9162396c9b (127.11.251.194:36383), ce4909f4936245de9d3ae7cdf436a23d (127.11.251.193:38593)
I20260927 18:38:55.653270 13683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106ea2eb57224cfbbbe957ef7e7fca2b" candidate_uuid: "908900369fd744718570aac5645c6bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f55e2417f3ad47a7997d0e9162396c9b"
I20260927 18:38:55.653290 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106ea2eb57224cfbbbe957ef7e7fca2b" candidate_uuid: "908900369fd744718570aac5645c6bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce4909f4936245de9d3ae7cdf436a23d"
I20260927 18:38:55.653344 13683 raft_consensus.cc:3037] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:55.653414 13543 raft_consensus.cc:3037] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:55.654163 13683 raft_consensus.cc:2454] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 908900369fd744718570aac5645c6bbe in term 1.
I20260927 18:38:55.654332 13756 leader_election.cc:302] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 908900369fd744718570aac5645c6bbe, f55e2417f3ad47a7997d0e9162396c9b; no voters:
I20260927 18:38:55.654444 13899 raft_consensus.cc:2781] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:55.654445 13543 raft_consensus.cc:2454] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 908900369fd744718570aac5645c6bbe in term 1.
I20260927 18:38:55.654613 13899 raft_consensus.cc:686] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe [term 1 LEADER]: Becoming Leader. State: Replica: 908900369fd744718570aac5645c6bbe, State: Running, Role: LEADER
I20260927 18:38:55.654721 13899 consensus_queue.cc:237] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:55.655539 13385 catalog_manager.cc:5495] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 908900369fd744718570aac5645c6bbe (127.11.251.195). New cstate: current_term: 1 leader_uuid: "908900369fd744718570aac5645c6bbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } health_report { overall_health: HEALTHY } } }
I20260927 18:38:55.684965 13890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.682322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53878 (local address 127.11.251.195:45447)
0927 18:38:55.682403 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:55.682405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:55.682415 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:38:55.682455 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:55.682456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:55.682471 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:55.682541 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:38:55.682693 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.683166 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.683981 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.684270 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.684356 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.684608 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:38:55.684704 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:38:55.684790 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:55.684834 (+ 44us) server_negotiation.cc:299] Negotiation successful
0927 18:38:55.684854 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 18:38:55.687270 13683 raft_consensus.cc:1261] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b [term 1 FOLLOWER]: Refusing update from remote peer 908900369fd744718570aac5645c6bbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:55.687629 13543 raft_consensus.cc:1261] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d [term 1 FOLLOWER]: Refusing update from remote peer 908900369fd744718570aac5645c6bbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:55.687733 13899 consensus_queue.cc:1035] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:55.687940 13902 consensus_queue.cc:1035] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:55.690670 13905 mvcc.cc:205] Tried to move back new op lower bound from 7334028638971609088 to 7334028638842224640. Current Snapshot: MvccSnapshot[applied={T|T < 7334028638971609088}]
I20260927 18:38:55.692837 13908 mvcc.cc:205] Tried to move back new op lower bound from 7334028638971609088 to 7334028638842224640. Current Snapshot: MvccSnapshot[applied={T|T < 7334028638971609088 or (T in {7334028638971609088})}]
I20260927 18:38:55.694917 13385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:55.695005 13385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:55.695858 13385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54: Sending DeleteTablet for 3 replicas of tablet 106ea2eb57224cfbbbe957ef7e7fca2b
I20260927 18:38:55.696231 13803 tablet_service.cc:1514] Processing DeleteTablet for tablet 106ea2eb57224cfbbbe957ef7e7fca2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:55 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:53874
I20260927 18:38:55.696239 13663 tablet_service.cc:1514] Processing DeleteTablet for tablet 106ea2eb57224cfbbbe957ef7e7fca2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:55 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:51188
I20260927 18:38:55.696348 13523 tablet_service.cc:1514] Processing DeleteTablet for tablet 106ea2eb57224cfbbbe957ef7e7fca2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:55 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:41336
I20260927 18:38:55.696537 13916 tablet_replica.cc:330] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d: stopping tablet replica
I20260927 18:38:55.696544 13914 tablet_replica.cc:330] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe: stopping tablet replica
I20260927 18:38:55.696638 13914 raft_consensus.cc:2229] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:38:55.696638 13916 raft_consensus.cc:2229] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:55.696733 13916 raft_consensus.cc:2258] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:55.696748 13914 raft_consensus.cc:2258] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:55.696641 13385 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:38:55.696847 13385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:38:55.697183 13914 ts_tablet_manager.cc:1905] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:55.697408 13916 ts_tablet_manager.cc:1905] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:55.698035 13915 tablet_replica.cc:330] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b: stopping tablet replica
I20260927 18:38:55.699275 13523 tablet_service.cc:1467] Processing CreateTablet for tablet 7d76507d565449d1a474d5b6d7e32d8c (DEFAULT_TABLE table=test-table [id=355f3d1b884f42d0b5f34c2b2f4386cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:55.699383 13523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d76507d565449d1a474d5b6d7e32d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:55.699416 13663 tablet_service.cc:1467] Processing CreateTablet for tablet 7d76507d565449d1a474d5b6d7e32d8c (DEFAULT_TABLE table=test-table [id=355f3d1b884f42d0b5f34c2b2f4386cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:55.699599 13663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d76507d565449d1a474d5b6d7e32d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:55.700027 13803 tablet_service.cc:1467] Processing CreateTablet for tablet 7d76507d565449d1a474d5b6d7e32d8c (DEFAULT_TABLE table=test-table [id=355f3d1b884f42d0b5f34c2b2f4386cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:55.700201 13803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d76507d565449d1a474d5b6d7e32d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:55.700650 13894 tablet_bootstrap.cc:492] T 7d76507d565449d1a474d5b6d7e32d8c P ce4909f4936245de9d3ae7cdf436a23d: Bootstrap starting.
I20260927 18:38:55.701017 13894 tablet_bootstrap.cc:654] T 7d76507d565449d1a474d5b6d7e32d8c P ce4909f4936245de9d3ae7cdf436a23d: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:55.701367 13893 tablet_bootstrap.cc:492] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b: Bootstrap starting.
I20260927 18:38:55.701393 13916 ts_tablet_manager.cc:1918] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:38:55.701460 13916 log.cc:1196] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-0/wal/wals/106ea2eb57224cfbbbe957ef7e7fca2b
I20260927 18:38:55.701546 13892 tablet_bootstrap.cc:492] T 7d76507d565449d1a474d5b6d7e32d8c P 908900369fd744718570aac5645c6bbe: Bootstrap starting.
I20260927 18:38:55.701784 13916 ts_tablet_manager.cc:1939] T 106ea2eb57224cfbbbe957ef7e7fca2b P ce4909f4936245de9d3ae7cdf436a23d: Deleting consensus metadata
I20260927 18:38:55.701884 13893 tablet_bootstrap.cc:654] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:55.701982 13892 tablet_bootstrap.cc:654] T 7d76507d565449d1a474d5b6d7e32d8c P 908900369fd744718570aac5645c6bbe: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:55.702366 13370 catalog_manager.cc:4841] TS ce4909f4936245de9d3ae7cdf436a23d (127.11.251.193:38593): tablet 106ea2eb57224cfbbbe957ef7e7fca2b (table test-table [id=4d3253fb05c249fa8091a506fb1ce21c]) successfully deleted
I20260927 18:38:55.702481 13894 tablet_bootstrap.cc:492] T 7d76507d565449d1a474d5b6d7e32d8c P ce4909f4936245de9d3ae7cdf436a23d: No bootstrap required, opened a new log
I20260927 18:38:55.702534 13894 ts_tablet_manager.cc:1397] T 7d76507d565449d1a474d5b6d7e32d8c P ce4909f4936245de9d3ae7cdf436a23d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:55.702679 13894 raft_consensus.cc:348] T 7d76507d565449d1a474d5b6d7e32d8c P ce4909f4936245de9d3ae7cdf436a23d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:55.702724 13914 ts_tablet_manager.cc:1918] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:38:55.702764 13894 raft_consensus.cc:374] T 7d76507d565449d1a474d5b6d7e32d8c P ce4909f4936245de9d3ae7cdf436a23d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:55.702786 13914 log.cc:1196] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/106ea2eb57224cfbbbe957ef7e7fca2b
I20260927 18:38:55.702786 13894 raft_consensus.cc:729] T 7d76507d565449d1a474d5b6d7e32d8c P ce4909f4936245de9d3ae7cdf436a23d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce4909f4936245de9d3ae7cdf436a23d, State: Initialized, Role: FOLLOWER
I20260927 18:38:55.702831 13894 consensus_queue.cc:260] T 7d76507d565449d1a474d5b6d7e32d8c P ce4909f4936245de9d3ae7cdf436a23d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:55.702941 13894 ts_tablet_manager.cc:1428] T 7d76507d565449d1a474d5b6d7e32d8c P ce4909f4936245de9d3ae7cdf436a23d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 18:38:55.703063 13914 ts_tablet_manager.cc:1939] T 106ea2eb57224cfbbbe957ef7e7fca2b P 908900369fd744718570aac5645c6bbe: Deleting consensus metadata
I20260927 18:38:55.703423 13372 catalog_manager.cc:4841] TS 908900369fd744718570aac5645c6bbe (127.11.251.195:45447): tablet 106ea2eb57224cfbbbe957ef7e7fca2b (table test-table [id=4d3253fb05c249fa8091a506fb1ce21c]) successfully deleted
I20260927 18:38:55.703603 13915 raft_consensus.cc:2229] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:55.703820 13915 raft_consensus.cc:2258] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:55.703841 13892 tablet_bootstrap.cc:492] T 7d76507d565449d1a474d5b6d7e32d8c P 908900369fd744718570aac5645c6bbe: No bootstrap required, opened a new log
I20260927 18:38:55.703902 13892 ts_tablet_manager.cc:1397] T 7d76507d565449d1a474d5b6d7e32d8c P 908900369fd744718570aac5645c6bbe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:55.704041 13892 raft_consensus.cc:348] T 7d76507d565449d1a474d5b6d7e32d8c P 908900369fd744718570aac5645c6bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:55.704109 13892 raft_consensus.cc:374] T 7d76507d565449d1a474d5b6d7e32d8c P 908900369fd744718570aac5645c6bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:55.704133 13892 raft_consensus.cc:729] T 7d76507d565449d1a474d5b6d7e32d8c P 908900369fd744718570aac5645c6bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 908900369fd744718570aac5645c6bbe, State: Initialized, Role: FOLLOWER
I20260927 18:38:55.704176 13892 consensus_queue.cc:260] T 7d76507d565449d1a474d5b6d7e32d8c P 908900369fd744718570aac5645c6bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:55.704272 13892 ts_tablet_manager.cc:1428] T 7d76507d565449d1a474d5b6d7e32d8c P 908900369fd744718570aac5645c6bbe: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 18:38:55.704484 13893 tablet_bootstrap.cc:492] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b: No bootstrap required, opened a new log
I20260927 18:38:55.704566 13893 ts_tablet_manager.cc:1397] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 18:38:55.704739 13915 ts_tablet_manager.cc:1905] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:55.704774 13893 raft_consensus.cc:348] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:55.704890 13893 raft_consensus.cc:374] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:55.704931 13893 raft_consensus.cc:729] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f55e2417f3ad47a7997d0e9162396c9b, State: Initialized, Role: FOLLOWER
I20260927 18:38:55.704993 13893 consensus_queue.cc:260] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:55.705184 13893 ts_tablet_manager.cc:1428] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:55.706713 13915 ts_tablet_manager.cc:1918] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:38:55.706799 13915 log.cc:1196] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/106ea2eb57224cfbbbe957ef7e7fca2b
I20260927 18:38:55.707027 13915 ts_tablet_manager.cc:1939] T 106ea2eb57224cfbbbe957ef7e7fca2b P f55e2417f3ad47a7997d0e9162396c9b: Deleting consensus metadata
I20260927 18:38:55.707444 13372 catalog_manager.cc:4841] TS f55e2417f3ad47a7997d0e9162396c9b (127.11.251.194:36383): tablet 106ea2eb57224cfbbbe957ef7e7fca2b (table test-table [id=4d3253fb05c249fa8091a506fb1ce21c]) successfully deleted
I20260927 18:38:55.949910 13900 raft_consensus.cc:482] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:38:55.950045 13900 raft_consensus.cc:504] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:55.950537 13900 leader_election.cc:288] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce4909f4936245de9d3ae7cdf436a23d (127.11.251.193:38593), 908900369fd744718570aac5645c6bbe (127.11.251.195:45447)
I20260927 18:38:55.954885 13890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.950980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:46425 (local address 127.11.251.195:45447)
0927 18:38:55.951036 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:55.951038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:55.951047 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:38:55.951085 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:55.951087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:55.951102 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:55.951209 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:55.951478 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.952350 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.954074 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.954539 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.954614 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.954624 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:55.954790 (+ 166us) server_negotiation.cc:299] Negotiation successful
0927 18:38:55.954823 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:38:55.954946 13921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.950783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:45447 (local address 127.11.251.194:46425)
0927 18:38:55.950929 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:55.950941 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:55.951004 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:55.951239 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:55.951240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:55.951243 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:55.951381 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.951385 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.952464 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.952466 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.953944 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.953946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.954632 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.954633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.954707 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.954712 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:38:55.954779 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 18:38:55.954827 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":87,"thread_start_us":73,"threads_started":1}
I20260927 18:38:55.955091 13920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.950785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:38593 (local address 127.11.251.194:59861)
0927 18:38:55.950929 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:55.950941 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:55.951005 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:55.951239 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:55.951240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:55.951244 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:55.951382 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.951385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.952475 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.952476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.953945 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:55.953946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.954839 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.954839 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:55.954899 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.954901 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:38:55.954956 (+ 55us) client_negotiation.cc:240] Negotiation successful
0927 18:38:55.954991 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":124,"thread_start_us":47,"threads_started":1}
I20260927 18:38:55.955274 13891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:55.950931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:59861 (local address 127.11.251.193:38593)
0927 18:38:55.950997 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:55.951000 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:55.951019 (+ 19us) server_negotiation.cc:407] Connection header received
0927 18:38:55.951092 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:55.951093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:55.951103 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:55.951209 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:55.951480 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.952389 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.954246 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:55.954701 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:55.954753 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:55.954764 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:55.955137 (+ 373us) server_negotiation.cc:299] Negotiation successful
0927 18:38:55.955195 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 18:38:55.955435 13823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d76507d565449d1a474d5b6d7e32d8c" candidate_uuid: "f55e2417f3ad47a7997d0e9162396c9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "908900369fd744718570aac5645c6bbe" is_pre_election: true
I20260927 18:38:55.955565 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d76507d565449d1a474d5b6d7e32d8c" candidate_uuid: "f55e2417f3ad47a7997d0e9162396c9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce4909f4936245de9d3ae7cdf436a23d" is_pre_election: true
I20260927 18:38:55.955627 13823 raft_consensus.cc:2454] T 7d76507d565449d1a474d5b6d7e32d8c P 908900369fd744718570aac5645c6bbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f55e2417f3ad47a7997d0e9162396c9b in term 0.
I20260927 18:38:55.955657 13543 raft_consensus.cc:2454] T 7d76507d565449d1a474d5b6d7e32d8c P ce4909f4936245de9d3ae7cdf436a23d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f55e2417f3ad47a7997d0e9162396c9b in term 0.
I20260927 18:38:55.955866 13614 leader_election.cc:302] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce4909f4936245de9d3ae7cdf436a23d, f55e2417f3ad47a7997d0e9162396c9b; no voters:
I20260927 18:38:55.956027 13900 raft_consensus.cc:2781] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:38:55.956092 13900 raft_consensus.cc:482] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:38:55.956115 13900 raft_consensus.cc:3037] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:55.957015 13900 raft_consensus.cc:504] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:55.957165 13900 leader_election.cc:288] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b [CANDIDATE]: Term 1 election: Requested vote from peers ce4909f4936245de9d3ae7cdf436a23d (127.11.251.193:38593), 908900369fd744718570aac5645c6bbe (127.11.251.195:45447)
I20260927 18:38:55.957351 13543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d76507d565449d1a474d5b6d7e32d8c" candidate_uuid: "f55e2417f3ad47a7997d0e9162396c9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce4909f4936245de9d3ae7cdf436a23d"
I20260927 18:38:55.957396 13823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d76507d565449d1a474d5b6d7e32d8c" candidate_uuid: "f55e2417f3ad47a7997d0e9162396c9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "908900369fd744718570aac5645c6bbe"
I20260927 18:38:55.957464 13823 raft_consensus.cc:3037] T 7d76507d565449d1a474d5b6d7e32d8c P 908900369fd744718570aac5645c6bbe [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:55.957472 13543 raft_consensus.cc:3037] T 7d76507d565449d1a474d5b6d7e32d8c P ce4909f4936245de9d3ae7cdf436a23d [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:55.958266 13823 raft_consensus.cc:2454] T 7d76507d565449d1a474d5b6d7e32d8c P 908900369fd744718570aac5645c6bbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f55e2417f3ad47a7997d0e9162396c9b in term 1.
I20260927 18:38:55.958460 13616 leader_election.cc:302] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 908900369fd744718570aac5645c6bbe, f55e2417f3ad47a7997d0e9162396c9b; no voters:
I20260927 18:38:55.958562 13900 raft_consensus.cc:2781] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:55.958591 13543 raft_consensus.cc:2454] T 7d76507d565449d1a474d5b6d7e32d8c P ce4909f4936245de9d3ae7cdf436a23d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f55e2417f3ad47a7997d0e9162396c9b in term 1.
I20260927 18:38:55.958734 13900 raft_consensus.cc:686] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b [term 1 LEADER]: Becoming Leader. State: Replica: f55e2417f3ad47a7997d0e9162396c9b, State: Running, Role: LEADER
I20260927 18:38:55.958819 13900 consensus_queue.cc:237] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:55.959618 13386 catalog_manager.cc:5495] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b reported cstate change: term changed from 0 to 1, leader changed from <none> to f55e2417f3ad47a7997d0e9162396c9b (127.11.251.194). New cstate: current_term: 1 leader_uuid: "f55e2417f3ad47a7997d0e9162396c9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } health_report { overall_health: UNKNOWN } } }
Sep 27 18:38:56 dist-test-slave-z47j krb5kdc[13341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534334, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:38:56.029253 13930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:56.014795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39336 (local address 127.11.251.254:45193)
0927 18:38:56.015001 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:56.015004 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:56.015169 (+ 165us) server_negotiation.cc:407] Connection header received
0927 18:38:56.015249 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:56.015250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:56.015262 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:56.015400 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:56.015986 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.016597 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.017332 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.017545 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.017667 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:56.026219 (+ 8552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:56.026221 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:56.026221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:56.026230 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:56.027783 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:56.028301 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:56.028302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:56.028302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:56.028374 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:56.028631 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:56.028631 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:56.028632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:56.028774 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:56.028840 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:56.029056 (+ 216us) server_negotiation.cc:299] Negotiation successful
0927 18:38:56.029157 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":114,"threads_started":1}
I20260927 18:38:56.029439 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:56.014704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:45193 (local address 127.0.0.1:39336)
0927 18:38:56.015031 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:56.015046 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:56.015177 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:56.015402 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:56.015403 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:56.015837 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:56.015894 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:56.015897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.016671 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.016672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:56.017233 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:56.017234 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.017676 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.017677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:56.017738 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:56.024168 (+ 6430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:56.024174 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:56.026088 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:56.027896 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:56.027903 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:56.027906 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:56.028198 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:56.028487 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:56.028488 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:56.028488 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:56.028558 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:56.028852 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:56.028859 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:56.028955 (+ 96us) client_negotiation.cc:769] Sending connection context
0927 18:38:56.029023 (+ 68us) client_negotiation.cc:240] Negotiation successful
0927 18:38:56.029281 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":105,"thread_start_us":163,"threads_started":1}
I20260927 18:38:56.032075 13386 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:39336:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 18:38:56.032284 13386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:38:56.034098 13803 tablet_service.cc:1467] Processing CreateTablet for tablet 775d8e6df78f4175813941bc2267f750 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=328f5ac2808d4fe2873c0e52f84c402e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:38:56.034133 13523 tablet_service.cc:1467] Processing CreateTablet for tablet 775d8e6df78f4175813941bc2267f750 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=328f5ac2808d4fe2873c0e52f84c402e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:38:56.034125 13663 tablet_service.cc:1467] Processing CreateTablet for tablet 775d8e6df78f4175813941bc2267f750 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=328f5ac2808d4fe2873c0e52f84c402e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:38:56.034265 13523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 775d8e6df78f4175813941bc2267f750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:56.034265 13803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 775d8e6df78f4175813941bc2267f750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:56.034426 13663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 775d8e6df78f4175813941bc2267f750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:56.035874 13894 tablet_bootstrap.cc:492] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d: Bootstrap starting.
I20260927 18:38:56.035912 13892 tablet_bootstrap.cc:492] T 775d8e6df78f4175813941bc2267f750 P 908900369fd744718570aac5645c6bbe: Bootstrap starting.
I20260927 18:38:56.036221 13893 tablet_bootstrap.cc:492] T 775d8e6df78f4175813941bc2267f750 P f55e2417f3ad47a7997d0e9162396c9b: Bootstrap starting.
I20260927 18:38:56.036255 13894 tablet_bootstrap.cc:654] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:56.036417 13892 tablet_bootstrap.cc:654] T 775d8e6df78f4175813941bc2267f750 P 908900369fd744718570aac5645c6bbe: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:56.036689 13893 tablet_bootstrap.cc:654] T 775d8e6df78f4175813941bc2267f750 P f55e2417f3ad47a7997d0e9162396c9b: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:56.036960 13894 tablet_bootstrap.cc:492] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d: No bootstrap required, opened a new log
I20260927 18:38:56.037015 13894 ts_tablet_manager.cc:1397] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:56.037252 13894 raft_consensus.cc:348] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:56.037361 13894 raft_consensus.cc:374] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:56.037395 13894 raft_consensus.cc:729] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce4909f4936245de9d3ae7cdf436a23d, State: Initialized, Role: FOLLOWER
I20260927 18:38:56.037389 13892 tablet_bootstrap.cc:492] T 775d8e6df78f4175813941bc2267f750 P 908900369fd744718570aac5645c6bbe: No bootstrap required, opened a new log
I20260927 18:38:56.037446 13892 ts_tablet_manager.cc:1397] T 775d8e6df78f4175813941bc2267f750 P 908900369fd744718570aac5645c6bbe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:56.037467 13894 consensus_queue.cc:260] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:56.037596 13892 raft_consensus.cc:348] T 775d8e6df78f4175813941bc2267f750 P 908900369fd744718570aac5645c6bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:56.037674 13892 raft_consensus.cc:374] T 775d8e6df78f4175813941bc2267f750 P 908900369fd744718570aac5645c6bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:56.037694 13892 raft_consensus.cc:729] T 775d8e6df78f4175813941bc2267f750 P 908900369fd744718570aac5645c6bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 908900369fd744718570aac5645c6bbe, State: Initialized, Role: FOLLOWER
I20260927 18:38:56.037758 13892 consensus_queue.cc:260] T 775d8e6df78f4175813941bc2267f750 P 908900369fd744718570aac5645c6bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:56.037736 13898 tablet_replica.cc:439] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } } }
I20260927 18:38:56.037806 13893 tablet_bootstrap.cc:492] T 775d8e6df78f4175813941bc2267f750 P f55e2417f3ad47a7997d0e9162396c9b: No bootstrap required, opened a new log
I20260927 18:38:56.037842 13898 tablet_replica.cc:442] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:38:56.037859 13893 ts_tablet_manager.cc:1397] T 775d8e6df78f4175813941bc2267f750 P f55e2417f3ad47a7997d0e9162396c9b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:56.037858 13892 ts_tablet_manager.cc:1428] T 775d8e6df78f4175813941bc2267f750 P 908900369fd744718570aac5645c6bbe: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 18:38:56.037866 13902 tablet_replica.cc:439] T 775d8e6df78f4175813941bc2267f750 P 908900369fd744718570aac5645c6bbe: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } } }
I20260927 18:38:56.037955 13902 tablet_replica.cc:442] T 775d8e6df78f4175813941bc2267f750 P 908900369fd744718570aac5645c6bbe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:38:56.038867 13893 raft_consensus.cc:348] T 775d8e6df78f4175813941bc2267f750 P f55e2417f3ad47a7997d0e9162396c9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:56.038969 13893 raft_consensus.cc:374] T 775d8e6df78f4175813941bc2267f750 P f55e2417f3ad47a7997d0e9162396c9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:56.038996 13893 raft_consensus.cc:729] T 775d8e6df78f4175813941bc2267f750 P f55e2417f3ad47a7997d0e9162396c9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f55e2417f3ad47a7997d0e9162396c9b, State: Initialized, Role: FOLLOWER
I20260927 18:38:56.039058 13893 consensus_queue.cc:260] T 775d8e6df78f4175813941bc2267f750 P f55e2417f3ad47a7997d0e9162396c9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:56.039194 13893 ts_tablet_manager.cc:1428] T 775d8e6df78f4175813941bc2267f750 P f55e2417f3ad47a7997d0e9162396c9b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260927 18:38:56.039229 13900 tablet_replica.cc:439] T 775d8e6df78f4175813941bc2267f750 P f55e2417f3ad47a7997d0e9162396c9b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } } }
I20260927 18:38:56.039335 13900 tablet_replica.cc:442] T 775d8e6df78f4175813941bc2267f750 P f55e2417f3ad47a7997d0e9162396c9b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:38:56.039392 13894 ts_tablet_manager.cc:1428] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:56.207134 13898 raft_consensus.cc:482] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:38:56.207301 13898 raft_consensus.cc:504] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:56.207755 13898 leader_election.cc:288] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f55e2417f3ad47a7997d0e9162396c9b (127.11.251.194:36383), 908900369fd744718570aac5645c6bbe (127.11.251.195:45447)
I20260927 18:38:56.211133 13890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:56.208244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:33679 (local address 127.11.251.195:45447)
0927 18:38:56.208295 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:56.208299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:56.208347 (+ 48us) server_negotiation.cc:407] Connection header received
0927 18:38:56.208458 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:56.208460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:56.208470 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:56.208538 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:56.208695 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.209271 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.210438 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.210795 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.210876 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:56.210886 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:56.211023 (+ 137us) server_negotiation.cc:299] Negotiation successful
0927 18:38:56.211058 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260927 18:38:56.211158 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:56.208178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:45447 (local address 127.11.251.193:33679)
0927 18:38:56.208287 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:56.208299 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:56.208369 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:56.208543 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:56.208545 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:56.208548 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:56.208612 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:56.208614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.209354 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.209355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:56.210331 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:56.210333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.210874 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.210875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:56.210943 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:56.210946 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:38:56.211006 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 18:38:56.211058 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":43,"threads_started":1}
I20260927 18:38:56.211210 13935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:56.208108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:42835 (local address 127.11.251.194:36383)
0927 18:38:56.208277 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:56.208279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:56.208297 (+ 18us) server_negotiation.cc:407] Connection header received
0927 18:38:56.208412 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:56.208413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:56.208429 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:56.208518 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:56.208693 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.209328 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.210430 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.210822 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.210919 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:56.210930 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:56.211076 (+ 146us) server_negotiation.cc:299] Negotiation successful
0927 18:38:56.211119 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":79,"threads_started":1}
I20260927 18:38:56.211278 13934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:56.207986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:36383 (local address 127.11.251.193:42835)
0927 18:38:56.208138 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:56.208158 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:56.208321 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:56.208526 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:56.208527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:56.208530 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:56.208624 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:56.208627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.209441 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.209442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:56.210336 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:56.210338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.210918 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.210919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:56.210996 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:56.211005 (+ 9us) client_negotiation.cc:769] Sending connection context
0927 18:38:56.211070 (+ 65us) client_negotiation.cc:240] Negotiation successful
0927 18:38:56.211122 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
I20260927 18:38:56.211375 13823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "775d8e6df78f4175813941bc2267f750" candidate_uuid: "ce4909f4936245de9d3ae7cdf436a23d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "908900369fd744718570aac5645c6bbe" is_pre_election: true
I20260927 18:38:56.211488 13823 raft_consensus.cc:2454] T 775d8e6df78f4175813941bc2267f750 P 908900369fd744718570aac5645c6bbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce4909f4936245de9d3ae7cdf436a23d in term 0.
I20260927 18:38:56.211520 13683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "775d8e6df78f4175813941bc2267f750" candidate_uuid: "ce4909f4936245de9d3ae7cdf436a23d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f55e2417f3ad47a7997d0e9162396c9b" is_pre_election: true
I20260927 18:38:56.211606 13683 raft_consensus.cc:2454] T 775d8e6df78f4175813941bc2267f750 P f55e2417f3ad47a7997d0e9162396c9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce4909f4936245de9d3ae7cdf436a23d in term 0.
I20260927 18:38:56.211673 13476 leader_election.cc:302] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 908900369fd744718570aac5645c6bbe, ce4909f4936245de9d3ae7cdf436a23d; no voters:
I20260927 18:38:56.211838 13898 raft_consensus.cc:2781] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:38:56.211891 13898 raft_consensus.cc:482] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:38:56.211920 13898 raft_consensus.cc:3037] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:56.212663 13898 raft_consensus.cc:504] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:56.212816 13898 leader_election.cc:288] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d [CANDIDATE]: Term 1 election: Requested vote from peers f55e2417f3ad47a7997d0e9162396c9b (127.11.251.194:36383), 908900369fd744718570aac5645c6bbe (127.11.251.195:45447)
I20260927 18:38:56.213027 13683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "775d8e6df78f4175813941bc2267f750" candidate_uuid: "ce4909f4936245de9d3ae7cdf436a23d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f55e2417f3ad47a7997d0e9162396c9b"
I20260927 18:38:56.213058 13823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "775d8e6df78f4175813941bc2267f750" candidate_uuid: "ce4909f4936245de9d3ae7cdf436a23d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "908900369fd744718570aac5645c6bbe"
I20260927 18:38:56.213130 13823 raft_consensus.cc:3037] T 775d8e6df78f4175813941bc2267f750 P 908900369fd744718570aac5645c6bbe [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:56.213137 13683 raft_consensus.cc:3037] T 775d8e6df78f4175813941bc2267f750 P f55e2417f3ad47a7997d0e9162396c9b [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:56.213888 13823 raft_consensus.cc:2454] T 775d8e6df78f4175813941bc2267f750 P 908900369fd744718570aac5645c6bbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce4909f4936245de9d3ae7cdf436a23d in term 1.
I20260927 18:38:56.214067 13476 leader_election.cc:302] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 908900369fd744718570aac5645c6bbe, ce4909f4936245de9d3ae7cdf436a23d; no voters:
I20260927 18:38:56.214104 13683 raft_consensus.cc:2454] T 775d8e6df78f4175813941bc2267f750 P f55e2417f3ad47a7997d0e9162396c9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce4909f4936245de9d3ae7cdf436a23d in term 1.
I20260927 18:38:56.214164 13898 raft_consensus.cc:2781] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:56.214303 13898 raft_consensus.cc:686] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d [term 1 LEADER]: Becoming Leader. State: Replica: ce4909f4936245de9d3ae7cdf436a23d, State: Running, Role: LEADER
I20260927 18:38:56.214392 13898 consensus_queue.cc:237] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } }
I20260927 18:38:56.214830 13937 tablet_replica.cc:439] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d: TxnStatusTablet state changed. Reason: New leader ce4909f4936245de9d3ae7cdf436a23d. Latest consensus state: current_term: 1 leader_uuid: "ce4909f4936245de9d3ae7cdf436a23d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } } }
I20260927 18:38:56.214938 13937 tablet_replica.cc:442] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:38:56.215167 13939 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 18:38:56.215272 13939 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 18:38:56.215283 13386 catalog_manager.cc:5495] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d reported cstate change: term changed from 0 to 1, leader changed from <none> to ce4909f4936245de9d3ae7cdf436a23d (127.11.251.193). New cstate: current_term: 1 leader_uuid: "ce4909f4936245de9d3ae7cdf436a23d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } health_report { overall_health: UNKNOWN } } }
I20260927 18:38:56.221436 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:56.218459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:38593 (local address 127.0.0.1:41348)
0927 18:38:56.218535 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:56.218546 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:56.218594 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:56.218778 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:56.218779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:56.218780 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:38:56.218821 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:56.218825 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.219834 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.219835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:56.220395 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:56.220396 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.220872 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.220873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:56.220917 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:56.220931 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:38:56.221246 (+ 315us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:38:56.221262 (+ 16us) client_negotiation.cc:769] Sending connection context
0927 18:38:56.221326 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 18:38:56.221370 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 18:38:56.221441 13891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:56.218515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41348 (local address 127.11.251.193:38593)
0927 18:38:56.218556 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:56.218559 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:56.218572 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:38:56.218682 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:56.218683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:56.218703 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:56.218780 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:38:56.218874 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.219736 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.220480 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.220788 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.220868 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:56.221024 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:38:56.221146 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:38:56.221239 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:56.221341 (+ 102us) server_negotiation.cc:299] Negotiation successful
0927 18:38:56.221361 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 18:38:56.222858 13823 raft_consensus.cc:1261] T 775d8e6df78f4175813941bc2267f750 P 908900369fd744718570aac5645c6bbe [term 1 FOLLOWER]: Refusing update from remote peer ce4909f4936245de9d3ae7cdf436a23d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:56.223085 13683 raft_consensus.cc:1261] T 775d8e6df78f4175813941bc2267f750 P f55e2417f3ad47a7997d0e9162396c9b [term 1 FOLLOWER]: Refusing update from remote peer ce4909f4936245de9d3ae7cdf436a23d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:56.223341 13937 consensus_queue.cc:1035] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:56.223449 13898 consensus_queue.cc:1035] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d [LEADER]: Connected to new peer: Peer: permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:56.224123 13900 tablet_replica.cc:439] T 775d8e6df78f4175813941bc2267f750 P f55e2417f3ad47a7997d0e9162396c9b: TxnStatusTablet state changed. Reason: New leader ce4909f4936245de9d3ae7cdf436a23d. Latest consensus state: current_term: 1 leader_uuid: "ce4909f4936245de9d3ae7cdf436a23d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } } }
I20260927 18:38:56.224229 13900 tablet_replica.cc:442] T 775d8e6df78f4175813941bc2267f750 P f55e2417f3ad47a7997d0e9162396c9b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:38:56.224220 13902 tablet_replica.cc:439] T 775d8e6df78f4175813941bc2267f750 P 908900369fd744718570aac5645c6bbe: TxnStatusTablet state changed. Reason: New leader ce4909f4936245de9d3ae7cdf436a23d. Latest consensus state: current_term: 1 leader_uuid: "ce4909f4936245de9d3ae7cdf436a23d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } } }
I20260927 18:38:56.224311 13902 tablet_replica.cc:442] T 775d8e6df78f4175813941bc2267f750 P 908900369fd744718570aac5645c6bbe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:38:56.225226 13942 mvcc.cc:205] Tried to move back new op lower bound from 7334028641166499840 to 7334028641134813184. Current Snapshot: MvccSnapshot[applied={T|T < 7334028641166499840}]
I20260927 18:38:56.225314 13937 tablet_replica.cc:439] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce4909f4936245de9d3ae7cdf436a23d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } } }
I20260927 18:38:56.225404 13937 tablet_replica.cc:442] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:38:56.225325 13898 tablet_replica.cc:439] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce4909f4936245de9d3ae7cdf436a23d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } } }
I20260927 18:38:56.225662 13898 tablet_replica.cc:442] T 775d8e6df78f4175813941bc2267f750 P ce4909f4936245de9d3ae7cdf436a23d: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:38:56.225630 13902 tablet_replica.cc:439] T 775d8e6df78f4175813941bc2267f750 P 908900369fd744718570aac5645c6bbe: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce4909f4936245de9d3ae7cdf436a23d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } } }
I20260927 18:38:56.225716 13902 tablet_replica.cc:442] T 775d8e6df78f4175813941bc2267f750 P 908900369fd744718570aac5645c6bbe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:38:56.226362 13900 tablet_replica.cc:439] T 775d8e6df78f4175813941bc2267f750 P f55e2417f3ad47a7997d0e9162396c9b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce4909f4936245de9d3ae7cdf436a23d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 } } peers { permanent_uuid: "f55e2417f3ad47a7997d0e9162396c9b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36383 } } peers { permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 } } }
I20260927 18:38:56.226449 13900 tablet_replica.cc:442] T 775d8e6df78f4175813941bc2267f750 P f55e2417f3ad47a7997d0e9162396c9b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:38:56.230661 13935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:56.227737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51196 (local address 127.11.251.194:36383)
0927 18:38:56.227785 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:56.227787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:56.227834 (+ 47us) server_negotiation.cc:407] Connection header received
0927 18:38:56.227907 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:56.227908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:56.227932 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:56.228017 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:38:56.228117 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.228852 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.229694 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.230016 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.230088 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:56.230235 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:38:56.230370 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:38:56.230449 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:56.230537 (+ 88us) server_negotiation.cc:299] Negotiation successful
0927 18:38:56.230562 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 18:38:56.235996 13891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:56.233218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:44743 (local address 127.11.251.193:38593)
0927 18:38:56.233286 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:56.233287 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:56.233379 (+ 92us) server_negotiation.cc:407] Connection header received
0927 18:38:56.233463 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:56.233464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:56.233471 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:56.233537 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:38:56.233703 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.234142 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.235282 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.235456 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.235518 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:56.235743 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:38:56.235773 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:38:56.235828 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:56.235905 (+ 77us) server_negotiation.cc:299] Negotiation successful
0927 18:38:56.235950 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 18:38:56.236035 13955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:56.233125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:38593 (local address 127.11.251.194:44743)
0927 18:38:56.233311 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:56.233324 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:56.233395 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:56.233555 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:56.233556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:56.233558 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:38:56.233644 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:56.233646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.234220 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.234222 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:56.235152 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:56.235154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.235537 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.235538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:56.235601 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:56.235616 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:38:56.235848 (+ 232us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:38:56.235850 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:38:56.235905 (+ 55us) client_negotiation.cc:240] Negotiation successful
0927 18:38:56.235952 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":97,"threads_started":1}
I20260927 18:38:56.238677 13543 raft_consensus.cc:1261] T 7d76507d565449d1a474d5b6d7e32d8c P ce4909f4936245de9d3ae7cdf436a23d [term 1 FOLLOWER]: Refusing update from remote peer f55e2417f3ad47a7997d0e9162396c9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:56.238962 13922 consensus_queue.cc:1035] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce4909f4936245de9d3ae7cdf436a23d" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:56.239488 13823 raft_consensus.cc:1261] T 7d76507d565449d1a474d5b6d7e32d8c P 908900369fd744718570aac5645c6bbe [term 1 FOLLOWER]: Refusing update from remote peer f55e2417f3ad47a7997d0e9162396c9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:56.239724 13922 consensus_queue.cc:1035] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "908900369fd744718570aac5645c6bbe" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:56.249009 13935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:56.246277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:51145 (local address 127.11.251.194:36383)
0927 18:38:56.246321 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:56.246323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:56.246430 (+ 107us) server_negotiation.cc:407] Connection header received
0927 18:38:56.246514 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:56.246514 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:56.246530 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:56.246577 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:38:56.246711 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.247165 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.248346 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.248524 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.248578 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:56.248793 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:38:56.248819 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:38:56.248855 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:56.248940 (+ 85us) server_negotiation.cc:299] Negotiation successful
0927 18:38:56.248957 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 18:38:56.249089 13960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:56.246099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:36383 (local address 127.11.251.193:51145)
0927 18:38:56.246356 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:56.246369 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:56.246439 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:56.246584 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:56.246585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:56.246586 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:38:56.246641 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:56.246643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.247239 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.247240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:56.248254 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:56.248256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.248609 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.248610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:56.248683 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:56.248698 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:38:56.248864 (+ 166us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:38:56.248866 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:38:56.248937 (+ 71us) client_negotiation.cc:240] Negotiation successful
0927 18:38:56.248978 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":113,"threads_started":1}
I20260927 18:38:56.258476 13385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:56.258613 13385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:56.259390 13385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 77cec5a02f4c4e1d837807a85391cf54: Sending DeleteTablet for 3 replicas of tablet 7d76507d565449d1a474d5b6d7e32d8c
I20260927 18:38:56.259755 13803 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d76507d565449d1a474d5b6d7e32d8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:56 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:53874
I20260927 18:38:56.259975 13663 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d76507d565449d1a474d5b6d7e32d8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:56 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:51188
I20260927 18:38:56.260296 13523 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d76507d565449d1a474d5b6d7e32d8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:56 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:41336
I20260927 18:38:56.260591 13965 tablet_replica.cc:330] T 7d76507d565449d1a474d5b6d7e32d8c P ce4909f4936245de9d3ae7cdf436a23d: stopping tablet replica
I20260927 18:38:56.260699 13965 raft_consensus.cc:2229] T 7d76507d565449d1a474d5b6d7e32d8c P ce4909f4936245de9d3ae7cdf436a23d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:56.260802 13965 raft_consensus.cc:2258] T 7d76507d565449d1a474d5b6d7e32d8c P ce4909f4936245de9d3ae7cdf436a23d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:56.261227 13965 ts_tablet_manager.cc:1905] T 7d76507d565449d1a474d5b6d7e32d8c P ce4909f4936245de9d3ae7cdf436a23d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:56.263523 13965 ts_tablet_manager.cc:1918] T 7d76507d565449d1a474d5b6d7e32d8c P ce4909f4936245de9d3ae7cdf436a23d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:38:56.263597 13965 log.cc:1196] T 7d76507d565449d1a474d5b6d7e32d8c P ce4909f4936245de9d3ae7cdf436a23d: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-0/wal/wals/7d76507d565449d1a474d5b6d7e32d8c
I20260927 18:38:56.263867 13965 ts_tablet_manager.cc:1939] T 7d76507d565449d1a474d5b6d7e32d8c P ce4909f4936245de9d3ae7cdf436a23d: Deleting consensus metadata
I20260927 18:38:56.264261 13370 catalog_manager.cc:4841] TS ce4909f4936245de9d3ae7cdf436a23d (127.11.251.193:38593): tablet 7d76507d565449d1a474d5b6d7e32d8c (table test-table [id=355f3d1b884f42d0b5f34c2b2f4386cc]) successfully deleted
I20260927 18:38:56.265712 13963 tablet_replica.cc:330] T 7d76507d565449d1a474d5b6d7e32d8c P 908900369fd744718570aac5645c6bbe: stopping tablet replica
I20260927 18:38:56.265739 13964 tablet_replica.cc:330] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b: stopping tablet replica
I20260927 18:38:56.265806 13963 raft_consensus.cc:2229] T 7d76507d565449d1a474d5b6d7e32d8c P 908900369fd744718570aac5645c6bbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:56.265806 13964 raft_consensus.cc:2229] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:38:56.265863 13963 raft_consensus.cc:2258] T 7d76507d565449d1a474d5b6d7e32d8c P 908900369fd744718570aac5645c6bbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:56.265929 13964 raft_consensus.cc:2258] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:56.266198 13963 ts_tablet_manager.cc:1905] T 7d76507d565449d1a474d5b6d7e32d8c P 908900369fd744718570aac5645c6bbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:56.266592 13964 ts_tablet_manager.cc:1905] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:56.268285 13963 ts_tablet_manager.cc:1918] T 7d76507d565449d1a474d5b6d7e32d8c P 908900369fd744718570aac5645c6bbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:38:56.268359 13963 log.cc:1196] T 7d76507d565449d1a474d5b6d7e32d8c P 908900369fd744718570aac5645c6bbe: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/7d76507d565449d1a474d5b6d7e32d8c
I20260927 18:38:56.268436 13964 ts_tablet_manager.cc:1918] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:38:56.268505 13964 log.cc:1196] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/7d76507d565449d1a474d5b6d7e32d8c
I20260927 18:38:56.268615 13963 ts_tablet_manager.cc:1939] T 7d76507d565449d1a474d5b6d7e32d8c P 908900369fd744718570aac5645c6bbe: Deleting consensus metadata
I20260927 18:38:56.268728 13964 ts_tablet_manager.cc:1939] T 7d76507d565449d1a474d5b6d7e32d8c P f55e2417f3ad47a7997d0e9162396c9b: Deleting consensus metadata
I20260927 18:38:56.269065 13372 catalog_manager.cc:4841] TS 908900369fd744718570aac5645c6bbe (127.11.251.195:45447): tablet 7d76507d565449d1a474d5b6d7e32d8c (table test-table [id=355f3d1b884f42d0b5f34c2b2f4386cc]) successfully deleted
I20260927 18:38:56.269173 13372 catalog_manager.cc:4841] TS f55e2417f3ad47a7997d0e9162396c9b (127.11.251.194:36383): tablet 7d76507d565449d1a474d5b6d7e32d8c (table test-table [id=355f3d1b884f42d0b5f34c2b2f4386cc]) successfully deleted
Sep 27 18:38:56 dist-test-slave-z47j krb5kdc[13341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534335, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.251.193@KRBTEST.COM
I20260927 18:38:56.280660 13891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:56.266336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41364 (local address 127.11.251.193:38593)
0927 18:38:56.266376 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:56.266379 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:56.269007 (+ 2628us) server_negotiation.cc:407] Connection header received
0927 18:38:56.269131 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:56.269132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:56.269145 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:56.269179 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:56.270078 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.270830 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.272345 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.272668 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.272751 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:56.276731 (+ 3980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:56.276747 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:56.276748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:56.276758 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:56.279017 (+ 2259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:56.279484 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:56.279488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:56.279488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:56.279562 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:56.279817 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:56.279817 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:56.279818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:56.280015 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:56.280197 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:56.280306 (+ 109us) server_negotiation.cc:299] Negotiation successful
0927 18:38:56.280515 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260927 18:38:56.280951 13583 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41364
I20260927 18:38:56.290660 13930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:56.282144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39348 (local address 127.11.251.254:45193)
0927 18:38:56.282194 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:56.282196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:56.282454 (+ 258us) server_negotiation.cc:407] Connection header received
0927 18:38:56.282521 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:56.282522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:56.282537 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:56.282609 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:56.283279 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.284097 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.285333 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:56.285720 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:56.285813 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:56.287445 (+ 1632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:56.287447 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:56.287447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:56.287455 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:56.289236 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:56.289740 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:56.289741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:56.289741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:56.289804 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:56.290045 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:56.290046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:56.290046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:56.290187 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:56.290303 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:56.290460 (+ 157us) server_negotiation.cc:299] Negotiation successful
0927 18:38:56.290558 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260927 18:38:56.291149 13385 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39348
I20260927 18:38:56.292026 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 13458
I20260927 18:38:56.298132 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 13598
I20260927 18:38:56.304462 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 13738
I20260927 18:38:56.311455 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 13354
2026-09-27T18:38:56Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2838 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:38:56 dist-test-slave-z47j krb5kdc[13972](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:38:56 dist-test-slave-z47j krb5kdc[13972](info): set up 2 sockets
Sep 27 18:38:56 dist-test-slave-z47j krb5kdc[13972](info): commencing operation
krb5kdc: starting...
W20260927 18:38:57.357920 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:38:57 dist-test-slave-z47j krb5kdc[13972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:38:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:38:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:38:57.484514 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:39005
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:35483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:39005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:57.565608 13985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:57.565819 13985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:57.565842 13985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:57.567000 13985 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:38:57.567045 13985 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 18:38:57.567059 13985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:57.567072 13985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:57.567085 13985 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:38:57.567152 13985 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:38:57.568596 13985 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:35483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:39005
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:39005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:38:57.568837 13985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:57.569074 13985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:57.574043 13990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:57.574205 13991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:57.574226 13993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:57.574568 13985 server_base.cc:1034] running on GCE node
I20260927 18:38:57.574905 13985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:57.575271 13985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:57.576450 13985 hybrid_clock.cc:648] HybridClock initialized: now 1790534337576428 us; error 52 us; skew 500 ppm
Sep 27 18:38:57 dist-test-slave-z47j krb5kdc[13972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534337, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:57.579288 13985 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:38:57.579804 13985 webserver.cc:460] Webserver started at http://127.11.251.254:37045/ using document root <none> and password file <none>
I20260927 18:38:57.580062 13985 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:57.580132 13985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:57.580287 13985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:57.581518 13985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "f895b58d92d549a5a016d57ef7750545"
format_stamp: "Formatted at 2026-09-27 18:38:57 on dist-test-slave-z47j"
server_key: "a031a9d4b1df78bdcb977ea1a961d4a2"
server_key_iv: "193bf6b5fa9e2291134b45ce7975355c"
server_key_version: "encryptionkey@0"
I20260927 18:38:57.581971 13985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "f895b58d92d549a5a016d57ef7750545"
format_stamp: "Formatted at 2026-09-27 18:38:57 on dist-test-slave-z47j"
server_key: "a031a9d4b1df78bdcb977ea1a961d4a2"
server_key_iv: "193bf6b5fa9e2291134b45ce7975355c"
server_key_version: "encryptionkey@0"
I20260927 18:38:57.583441 13985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:57.584295 14000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:57.584558 13985 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:57.584676 13985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "f895b58d92d549a5a016d57ef7750545"
format_stamp: "Formatted at 2026-09-27 18:38:57 on dist-test-slave-z47j"
server_key: "a031a9d4b1df78bdcb977ea1a961d4a2"
server_key_iv: "193bf6b5fa9e2291134b45ce7975355c"
server_key_version: "encryptionkey@0"
I20260927 18:38:57.584750 13985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:57.598428 13985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:57.600737 13985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:57.600930 13985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:57.606630 13985 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:39005
I20260927 18:38:57.606693 14062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:39005 every 8 connection(s)
I20260927 18:38:57.607079 13985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:38:57.607785 14063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:57.609834 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 13985
I20260927 18:38:57.609936 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:38:57.610140 12271 external_mini_cluster.cc:1428] Setting key 8a1b83fe9bf55297e1bd548b834bfe88
I20260927 18:38:57.610467 14063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545: Bootstrap starting.
I20260927 18:38:57.611397 14063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:57.611704 14063 log.cc:824] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:57.612386 14063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545: No bootstrap required, opened a new log
I20260927 18:38:57.614496 14063 raft_consensus.cc:348] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f895b58d92d549a5a016d57ef7750545" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 39005 } }
I20260927 18:38:57.614673 14063 raft_consensus.cc:374] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:57.614718 14063 raft_consensus.cc:729] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f895b58d92d549a5a016d57ef7750545, State: Initialized, Role: FOLLOWER
I20260927 18:38:57.614873 14063 consensus_queue.cc:260] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f895b58d92d549a5a016d57ef7750545" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 39005 } }
I20260927 18:38:57.614959 14063 raft_consensus.cc:388] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:38:57.615000 14063 raft_consensus.cc:482] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:38:57.615047 14063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:57.615630 14063 raft_consensus.cc:504] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f895b58d92d549a5a016d57ef7750545" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 39005 } }
I20260927 18:38:57.615747 14063 leader_election.cc:302] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f895b58d92d549a5a016d57ef7750545; no voters:
I20260927 18:38:57.615926 14063 leader_election.cc:288] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:38:57.616117 14069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:57.616176 14063 sys_catalog.cc:564] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:38:57.616331 14069 raft_consensus.cc:686] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545 [term 1 LEADER]: Becoming Leader. State: Replica: f895b58d92d549a5a016d57ef7750545, State: Running, Role: LEADER
I20260927 18:38:57.616449 14069 consensus_queue.cc:237] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f895b58d92d549a5a016d57ef7750545" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 39005 } }
I20260927 18:38:57.616895 14069 sys_catalog.cc:455] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f895b58d92d549a5a016d57ef7750545. Latest consensus state: current_term: 1 leader_uuid: "f895b58d92d549a5a016d57ef7750545" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f895b58d92d549a5a016d57ef7750545" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 39005 } } }
I20260927 18:38:57.616991 14069 sys_catalog.cc:458] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:57.617025 14068 sys_catalog.cc:455] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f895b58d92d549a5a016d57ef7750545" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f895b58d92d549a5a016d57ef7750545" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 39005 } } }
I20260927 18:38:57.617084 14068 sys_catalog.cc:458] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:57.617985 14076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:38:57.618515 14076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 27 18:38:57 dist-test-slave-z47j krb5kdc[13972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:38:57.620309 14076 catalog_manager.cc:1347] Generated new cluster ID: 1dd0f1b1bcc5406d9166097f6fb06a9f
I20260927 18:38:57.620364 14076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:38:57.623394 14066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:57.610721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52140 (local address 127.11.251.254:39005)
0927 18:38:57.610972 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:57.610981 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:57.611017 (+ 36us) server_negotiation.cc:407] Connection header received
0927 18:38:57.611730 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:57.611739 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:57.611815 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:57.612011 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:57.612832 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:57.613998 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:57.614841 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:57.615407 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:57.615508 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:57.619289 (+ 3781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:57.619295 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:57.619295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:57.619311 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:57.621518 (+ 2207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:57.622065 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:57.622068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:57.622068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:57.622181 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:57.622428 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:57.622429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:57.622429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:57.622725 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:57.622868 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:57.623055 (+ 187us) server_negotiation.cc:299] Negotiation successful
0927 18:38:57.623194 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":111,"threads_started":1}
I20260927 18:38:57.632884 14076 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:38:57.633817 14076 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:38:57.642266 14076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545: Generated new TSK 0
I20260927 18:38:57.642491 14076 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.251.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
I20260927 18:38:57.670847 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:39005
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:35483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:57.753356 14089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:57.753557 14089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:57.753643 14089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:57.754824 14089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:57.754865 14089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:57.754886 14089 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 18:38:57.754920 14089 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:38:57.754963 14089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:38:57.756386 14089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:35483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.251.254:39005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:38:57.756623 14089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:57.756855 14089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:57.762046 14097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:57.762063 14094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:57.762063 14095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:57.762441 14089 server_base.cc:1034] running on GCE node
I20260927 18:38:57.762676 14089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:57.762923 14089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:57.764114 14089 hybrid_clock.cc:648] HybridClock initialized: now 1790534337764104 us; error 46 us; skew 500 ppm
Sep 27 18:38:57 dist-test-slave-z47j krb5kdc[13972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534337, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:57.766786 14089 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:38:57.767293 14089 webserver.cc:460] Webserver started at http://127.11.251.193:38313/ using document root <none> and password file <none>
I20260927 18:38:57.767514 14089 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:57.767580 14089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:57.767727 14089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:57.768754 14089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "0be07e30136e46bba4bd70ed021b5fbe"
format_stamp: "Formatted at 2026-09-27 18:38:57 on dist-test-slave-z47j"
server_key: "6144258c9ed8d3081b8ba553a92dc7f9"
server_key_iv: "321ab07cb59d509a93d0a69c776f0742"
server_key_version: "encryptionkey@0"
I20260927 18:38:57.769121 14089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "0be07e30136e46bba4bd70ed021b5fbe"
format_stamp: "Formatted at 2026-09-27 18:38:57 on dist-test-slave-z47j"
server_key: "6144258c9ed8d3081b8ba553a92dc7f9"
server_key_iv: "321ab07cb59d509a93d0a69c776f0742"
server_key_version: "encryptionkey@0"
I20260927 18:38:57.770642 14089 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:38:57.771492 14104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:57.771777 14089 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:57.771911 14089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "0be07e30136e46bba4bd70ed021b5fbe"
format_stamp: "Formatted at 2026-09-27 18:38:57 on dist-test-slave-z47j"
server_key: "6144258c9ed8d3081b8ba553a92dc7f9"
server_key_iv: "321ab07cb59d509a93d0a69c776f0742"
server_key_version: "encryptionkey@0"
I20260927 18:38:57.772045 14089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:57.790283 14089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:57.792397 14089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:57.792568 14089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:57.793207 14089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:57.793267 14089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:57.793306 14089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:57.793321 14089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:38:57 dist-test-slave-z47j krb5kdc[13972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534337, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:38:57.801455 14089 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:39889
I20260927 18:38:57.801501 14219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:39889 every 8 connection(s)
I20260927 18:38:57.801895 14089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:38:57.804383 14066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:57.793808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:47117 (local address 127.11.251.254:39005)
0927 18:38:57.793893 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:57.793896 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:57.794274 (+ 378us) server_negotiation.cc:407] Connection header received
0927 18:38:57.795071 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:57.795073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:57.795091 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:57.795178 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:57.796135 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:57.796696 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:57.798446 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:57.798690 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:57.798757 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:57.801370 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:57.801372 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:57.801372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:57.801381 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:57.803017 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:57.803464 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:57.803465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:57.803466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:57.803542 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:57.803793 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:57.803793 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:57.803793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:57.803908 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:57.803963 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:57.804231 (+ 268us) server_negotiation.cc:299] Negotiation successful
0927 18:38:57.804317 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:38:57.804590 14113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:57.793809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:39005 (local address 127.11.251.193:47117)
0927 18:38:57.794081 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:57.794137 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:57.794905 (+ 768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:57.795268 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:57.795272 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:57.795714 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:57.796014 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:57.796020 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:57.796825 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:57.796827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:57.798322 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:57.798326 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:57.798781 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:57.798782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:57.798957 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:57.799579 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:57.799590 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:57.801281 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:57.803107 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:57.803113 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:57.803115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:57.803373 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:57.803611 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:57.803611 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:57.803611 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:57.803731 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:57.803968 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:57.803972 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:57.804109 (+ 137us) client_negotiation.cc:769] Sending connection context
0927 18:38:57.804204 (+ 95us) client_negotiation.cc:240] Negotiation successful
0927 18:38:57.804353 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":151,"threads_started":1}
I20260927 18:38:57.805265 14220 heartbeater.cc:347] Connected to a master server at 127.11.251.254:39005
I20260927 18:38:57.805380 14220 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:57.805625 14220 heartbeater.cc:511] Master 127.11.251.254:39005 requested a full tablet report, sending...
I20260927 18:38:57.805810 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 14089
I20260927 18:38:57.805917 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:38:57.806129 12271 external_mini_cluster.cc:1428] Setting key 4b6e0fa6b4f2f92231a18f798307edd3
I20260927 18:38:57.806311 14016 ts_manager.cc:194] Registered new tserver with Master: 0be07e30136e46bba4bd70ed021b5fbe (127.11.251.193:39889)
I20260927 18:38:57.807297 14016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:47117
I20260927 18:38:57.814924 14226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:57.806619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:39005 (local address 127.11.251.193:58317)
0927 18:38:57.807577 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:57.807591 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:57.807669 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:57.808481 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:57.808482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:57.808746 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:57.808843 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:57.808845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:57.809853 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:57.809854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:57.810392 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:57.810393 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:57.810889 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:57.810890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:57.810962 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:57.811549 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:57.811554 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:57.811907 (+ 353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:57.813882 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:57.813883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:57.813884 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:57.814062 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:57.814288 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:57.814288 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:57.814289 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:57.814354 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:57.814644 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:57.814645 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:57.814683 (+ 38us) client_negotiation.cc:769] Sending connection context
0927 18:38:57.814748 (+ 65us) client_negotiation.cc:240] Negotiation successful
0927 18:38:57.814837 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":924,"thread_start_us":135,"threads_started":1}
I20260927 18:38:57.815028 14066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:57.806540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:58317 (local address 127.11.251.254:39005)
0927 18:38:57.806576 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:57.806578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:57.807669 (+ 1091us) server_negotiation.cc:407] Connection header received
0927 18:38:57.807752 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:57.807753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:57.807764 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:57.807803 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:57.808927 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:57.809736 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:57.810483 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:57.810802 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:57.810883 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:57.812019 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:57.812021 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:57.812021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:57.812029 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:57.813781 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:57.814139 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:57.814140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:57.814141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:57.814204 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:57.814428 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:57.814429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:57.814430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:57.814567 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:57.814647 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:57.814799 (+ 152us) server_negotiation.cc:299] Negotiation successful
0927 18:38:57.814895 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for kudu/127.11.251.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
I20260927 18:38:57.837214 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:39005
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:35483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:57.919759 14229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:57.919965 14229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:57.919989 14229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:57.921113 14229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:57.921159 14229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:57.921180 14229 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 18:38:57.921191 14229 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:38:57.921208 14229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:38:57.922648 14229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:35483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.251.254:39005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:38:57.922880 14229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:57.923133 14229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:57.928296 14234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:57.928388 14235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:57.928480 14229 server_base.cc:1034] running on GCE node
W20260927 18:38:57.928299 14237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:57.928905 14229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:57.929243 14229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:57.930450 14229 hybrid_clock.cc:648] HybridClock initialized: now 1790534337930424 us; error 50 us; skew 500 ppm
Sep 27 18:38:57 dist-test-slave-z47j krb5kdc[13972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534337, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:57.933782 14229 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:38:57.934332 14229 webserver.cc:460] Webserver started at http://127.11.251.194:32889/ using document root <none> and password file <none>
I20260927 18:38:57.934626 14229 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:57.934700 14229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:57.934878 14229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:57.936216 14229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "06231b35aef94c21aa17f67a4ce44db6"
format_stamp: "Formatted at 2026-09-27 18:38:57 on dist-test-slave-z47j"
server_key: "d2effafa71fade3940f9958daba9fcca"
server_key_iv: "7a012e468e14c936226777298f601f95"
server_key_version: "encryptionkey@0"
I20260927 18:38:57.936640 14229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "06231b35aef94c21aa17f67a4ce44db6"
format_stamp: "Formatted at 2026-09-27 18:38:57 on dist-test-slave-z47j"
server_key: "d2effafa71fade3940f9958daba9fcca"
server_key_iv: "7a012e468e14c936226777298f601f95"
server_key_version: "encryptionkey@0"
I20260927 18:38:57.938354 14229 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260927 18:38:57.939280 14244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:57.939504 14229 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:57.939620 14229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "06231b35aef94c21aa17f67a4ce44db6"
format_stamp: "Formatted at 2026-09-27 18:38:57 on dist-test-slave-z47j"
server_key: "d2effafa71fade3940f9958daba9fcca"
server_key_iv: "7a012e468e14c936226777298f601f95"
server_key_version: "encryptionkey@0"
I20260927 18:38:57.939702 14229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:57.950410 14229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:57.952824 14229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:57.952996 14229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:57.953567 14229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:57.953678 14229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:57.953735 14229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:57.953758 14229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:38:57 dist-test-slave-z47j krb5kdc[13972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534337, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:38:57.962520 14229 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:41337
I20260927 18:38:57.962646 14359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:41337 every 8 connection(s)
I20260927 18:38:57.963050 14229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:38:57.965322 14066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:57.954218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:48873 (local address 127.11.251.254:39005)
0927 18:38:57.954311 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:57.954315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:57.954515 (+ 200us) server_negotiation.cc:407] Connection header received
0927 18:38:57.955056 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:57.955058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:57.955072 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:57.955171 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:57.955952 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:57.956732 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:57.958644 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:57.958952 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:57.959022 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:57.962160 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:57.962162 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:57.962162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:57.962169 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:57.963798 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:57.964271 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:57.964272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:57.964272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:57.964329 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:57.964588 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:57.964589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:57.964589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:57.964780 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:57.964862 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:57.965145 (+ 283us) server_negotiation.cc:299] Negotiation successful
0927 18:38:57.965223 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 18:38:57.965530 14253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:57.954142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:39005 (local address 127.11.251.194:48873)
0927 18:38:57.954363 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:57.954408 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:57.954939 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:57.955252 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:57.955256 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:57.955521 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:57.955858 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:57.955863 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:57.956856 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:57.956858 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:57.958514 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:57.958518 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:57.959065 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:57.959067 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:57.959208 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:57.959983 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:57.959991 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:57.962052 (+ 2061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:57.963902 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:57.963908 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:57.963911 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:57.964196 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:57.964402 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:57.964403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:57.964403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:57.964511 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:57.964865 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:57.964872 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:57.965017 (+ 145us) client_negotiation.cc:769] Sending connection context
0927 18:38:57.965122 (+ 105us) client_negotiation.cc:240] Negotiation successful
0927 18:38:57.965293 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":127,"threads_started":1}
I20260927 18:38:57.966169 14360 heartbeater.cc:347] Connected to a master server at 127.11.251.254:39005
I20260927 18:38:57.966287 14360 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:57.966495 14360 heartbeater.cc:511] Master 127.11.251.254:39005 requested a full tablet report, sending...
I20260927 18:38:57.966922 14017 ts_manager.cc:194] Registered new tserver with Master: 06231b35aef94c21aa17f67a4ce44db6 (127.11.251.194:41337)
I20260927 18:38:57.967476 14017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:48873
I20260927 18:38:57.973348 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 14229
I20260927 18:38:57.973476 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:38:57.973714 14066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:57.967340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:54375 (local address 127.11.251.254:39005)
0927 18:38:57.967392 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:57.967394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:57.967494 (+ 100us) server_negotiation.cc:407] Connection header received
0927 18:38:57.967633 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:57.967635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:57.967647 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:57.967731 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:57.968223 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:57.968668 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:57.969443 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:57.969716 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:57.969793 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:57.970948 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:57.970949 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:57.970949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:57.970955 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:57.972332 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:57.972703 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:57.972704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:57.972704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:57.972753 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:57.972969 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:57.972970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:57.972971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:57.973122 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:57.973198 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:57.973539 (+ 341us) server_negotiation.cc:299] Negotiation successful
0927 18:38:57.973657 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 18:38:57.973744 14366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:57.967228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:39005 (local address 127.11.251.194:54375)
0927 18:38:57.967392 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:57.967407 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:57.967494 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:57.967736 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:57.967737 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:57.968038 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:57.968132 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:57.968134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:57.968780 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:57.968782 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:57.969349 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:57.969351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:57.969804 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:57.969805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:57.969870 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:57.970361 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:57.970366 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:57.970826 (+ 460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:57.972435 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:57.972436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:57.972437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:57.972634 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:57.972820 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:57.972820 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:57.972821 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:57.972879 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:57.973237 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:57.973239 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:57.973311 (+ 72us) client_negotiation.cc:769] Sending connection context
0927 18:38:57.973429 (+ 118us) client_negotiation.cc:240] Negotiation successful
0927 18:38:57.973599 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":74,"threads_started":1}
I20260927 18:38:57.973757 12271 external_mini_cluster.cc:1428] Setting key f8c5d0d05bd0f4136ad3bfa78183d6e0
WARNING: no policy specified for kudu/127.11.251.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
I20260927 18:38:58.001796 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:39005
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:35483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:58.083401 14369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:58.083621 14369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:58.083658 14369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:58.084800 14369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:58.084860 14369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:58.084895 14369 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 18:38:58.084924 14369 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:38:58.084954 14369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:38:58.086401 14369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:35483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.251.254:39005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:38:58.086647 14369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:58.086858 14369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:58.091841 14374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:58.091868 14377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:58.091868 14375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:58.092142 14369 server_base.cc:1034] running on GCE node
I20260927 18:38:58.092341 14369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:58.092588 14369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:58.093717 14369 hybrid_clock.cc:648] HybridClock initialized: now 1790534338093705 us; error 33 us; skew 500 ppm
Sep 27 18:38:58 dist-test-slave-z47j krb5kdc[13972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534338, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:58.096125 14369 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:38:58.096566 14369 webserver.cc:460] Webserver started at http://127.11.251.195:33869/ using document root <none> and password file <none>
I20260927 18:38:58.096793 14369 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:58.096843 14369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:58.096931 14369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:58.097931 14369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "96f4d38fc01f453b9f5369a816fc3c59"
format_stamp: "Formatted at 2026-09-27 18:38:58 on dist-test-slave-z47j"
server_key: "8c58a3ec12ca258045d44a5022ab8a52"
server_key_iv: "c3e0798b363b1da224d02ea1101f72bc"
server_key_version: "encryptionkey@0"
I20260927 18:38:58.098232 14369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "96f4d38fc01f453b9f5369a816fc3c59"
format_stamp: "Formatted at 2026-09-27 18:38:58 on dist-test-slave-z47j"
server_key: "8c58a3ec12ca258045d44a5022ab8a52"
server_key_iv: "c3e0798b363b1da224d02ea1101f72bc"
server_key_version: "encryptionkey@0"
I20260927 18:38:58.099527 14369 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:38:58.100250 14384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:58.100582 14369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:58.100716 14369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "96f4d38fc01f453b9f5369a816fc3c59"
format_stamp: "Formatted at 2026-09-27 18:38:58 on dist-test-slave-z47j"
server_key: "8c58a3ec12ca258045d44a5022ab8a52"
server_key_iv: "c3e0798b363b1da224d02ea1101f72bc"
server_key_version: "encryptionkey@0"
I20260927 18:38:58.100800 14369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:58.110113 14369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:58.112079 14369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:58.112262 14369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:58.112826 14369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:58.112890 14369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:58.112938 14369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:58.112965 14369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:38:58 dist-test-slave-z47j krb5kdc[13972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534338, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:38:58.122011 14369 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:38547
I20260927 18:38:58.122087 14499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:38547 every 8 connection(s)
I20260927 18:38:58.122381 14369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:38:58.125150 14066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.113291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:35251 (local address 127.11.251.254:39005)
0927 18:38:58.113375 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:58.113378 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:58.113684 (+ 306us) server_negotiation.cc:407] Connection header received
0927 18:38:58.114564 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:58.114565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:58.114698 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:58.114792 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:58.115816 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.116349 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.118209 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.118560 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.118650 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.122076 (+ 3426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:58.122077 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:58.122078 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:58.122085 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:58.123813 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:58.124249 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:58.124250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:58.124250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:58.124300 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:58.124552 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:58.124552 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:58.124553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:58.124693 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:58.124766 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:58.125014 (+ 248us) server_negotiation.cc:299] Negotiation successful
0927 18:38:58.125078 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 18:38:58.125429 14393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.113251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:39005 (local address 127.11.251.195:35251)
0927 18:38:58.113476 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:58.113536 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:58.114422 (+ 886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:58.114894 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:58.114900 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:58.115340 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:58.115684 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.115690 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.116442 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.116443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.118098 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.118101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.118690 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.118691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.118867 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.119498 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:58.119508 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:58.121940 (+ 2432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:58.123906 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:58.123912 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:58.123914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:58.124161 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:58.124364 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:58.124365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:58.124365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:58.124470 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:58.124765 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:58.124770 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:58.124903 (+ 133us) client_negotiation.cc:769] Sending connection context
0927 18:38:58.124988 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 18:38:58.125125 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":113,"threads_started":1}
I20260927 18:38:58.126019 14500 heartbeater.cc:347] Connected to a master server at 127.11.251.254:39005
I20260927 18:38:58.126127 14500 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:58.126332 14500 heartbeater.cc:511] Master 127.11.251.254:39005 requested a full tablet report, sending...
I20260927 18:38:58.126748 14017 ts_manager.cc:194] Registered new tserver with Master: 96f4d38fc01f453b9f5369a816fc3c59 (127.11.251.195:38547)
I20260927 18:38:58.127216 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 14369
I20260927 18:38:58.127313 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:38:58.127312 14017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:35251
I20260927 18:38:58.127545 12271 external_mini_cluster.cc:1428] Setting key a67289c638e00faa6ffe607a0881a078
I20260927 18:38:58.128582 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:38:58.133622 14506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.127447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:39005 (local address 127.11.251.195:35317)
0927 18:38:58.127617 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:58.127631 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:58.127726 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:58.127860 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:58.127861 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:58.128092 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:58.128213 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.128216 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.128939 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.128940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.129488 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.129490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.129899 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.129900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.129991 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.130492 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:58.130497 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:58.130822 (+ 325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:58.132564 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:58.132565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:58.132566 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:58.132739 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:58.132947 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:58.132948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:58.132948 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:58.133006 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:58.133250 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:58.133251 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:58.133298 (+ 47us) client_negotiation.cc:769] Sending connection context
0927 18:38:58.133381 (+ 83us) client_negotiation.cc:240] Negotiation successful
0927 18:38:58.133491 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260927 18:38:58.133616 14066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.127370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:35317 (local address 127.11.251.254:39005)
0927 18:38:58.127409 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:58.127411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:58.127712 (+ 301us) server_negotiation.cc:407] Connection header received
0927 18:38:58.127792 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:58.127793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:58.127804 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:58.127856 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:58.128286 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.128850 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.129604 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.129816 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.129899 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.130934 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:58.130936 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:58.130936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:58.130943 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:58.132456 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:58.132819 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:58.132820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:58.132821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:58.132877 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:58.133074 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:58.133075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:58.133076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:58.133187 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:58.133260 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:58.133406 (+ 146us) server_negotiation.cc:299] Negotiation successful
0927 18:38:58.133486 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 27 18:38:58 dist-test-slave-z47j krb5kdc[13972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 18:38:58 dist-test-slave-z47j krb5kdc[13972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:38:58.151845 14066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.144541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52154 (local address 127.11.251.254:39005)
0927 18:38:58.144605 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:58.144607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:58.144658 (+ 51us) server_negotiation.cc:407] Connection header received
0927 18:38:58.144739 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:58.144740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:58.144749 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:58.144799 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:58.145434 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.145992 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.146798 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.146990 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.147065 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.149481 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:58.149483 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:58.149483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:58.149490 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:58.150849 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:58.151187 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:58.151188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:58.151188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:58.151240 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:58.151418 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:58.151418 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:58.151419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:58.151518 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:58.151606 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:58.151716 (+ 110us) server_negotiation.cc:299] Negotiation successful
0927 18:38:58.151779 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 18:38:58.153538 14017 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52154:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:38:58.155288 14017 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:38:58.162285 14519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.157884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:41337 (local address 127.0.0.1:33680)
0927 18:38:58.158290 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:58.158300 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:58.158341 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:58.158911 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:58.158912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:58.158915 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:58.158959 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.158960 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.160246 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.160247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.161226 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.161228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.161955 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.161956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.162034 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.162049 (+ 15us) client_negotiation.cc:769] Sending connection context
0927 18:38:58.162145 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 18:38:58.162204 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":107,"threads_started":1}
I20260927 18:38:58.162298 14522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.158164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33680 (local address 127.11.251.194:41337)
0927 18:38:58.158364 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:58.158370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:58.158380 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:38:58.158431 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:58.158433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:58.158446 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:58.158851 (+ 405us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:58.159083 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.160074 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.161307 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.161874 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.161960 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.162025 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:58.162162 (+ 137us) server_negotiation.cc:299] Negotiation successful
0927 18:38:58.162204 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":133,"threads_started":1}
I20260927 18:38:58.162638 14521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.158130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51706 (local address 127.11.251.193:39889)
0927 18:38:58.158457 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:58.158461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:58.158470 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:38:58.158517 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:58.158519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:58.158533 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:58.158604 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:58.158752 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.159623 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.161331 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.162125 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.162205 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.162275 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:58.162472 (+ 197us) server_negotiation.cc:299] Negotiation successful
0927 18:38:58.162546 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":78,"threads_started":1}
I20260927 18:38:58.162482 14517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.157848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:38547 (local address 127.0.0.1:47012)
0927 18:38:58.158034 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:58.158078 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:58.158154 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:58.158524 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:58.158527 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:58.158530 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:58.158613 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.158616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.159824 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.159825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.161231 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.161232 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.162231 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.162232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.162303 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.162308 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:38:58.162338 (+ 30us) client_negotiation.cc:240] Negotiation successful
0927 18:38:58.162393 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":78,"thread_start_us":60,"threads_started":1}
I20260927 18:38:58.162575 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.157866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:39889 (local address 127.0.0.1:51706)
0927 18:38:58.158127 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:58.158139 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:58.158201 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:58.158604 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:58.158606 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:58.158609 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:58.158665 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.158667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.159711 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.159712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.161224 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.161227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.162310 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.162311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.162370 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.162373 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:38:58.162427 (+ 54us) client_negotiation.cc:240] Negotiation successful
0927 18:38:58.162467 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":66,"thread_start_us":93,"threads_started":1}
I20260927 18:38:58.162932 14520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.157946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47012 (local address 127.11.251.195:38547)
0927 18:38:58.158325 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:58.158330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:58.158342 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:38:58.158386 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:58.158388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:58.158404 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:58.158501 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:58.158691 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.159736 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.161338 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.162125 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.162623 (+ 498us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.162677 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:58.162723 (+ 46us) server_negotiation.cc:299] Negotiation successful
0927 18:38:58.162794 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":91,"threads_started":1}
I20260927 18:38:58.163650 14154 tablet_service.cc:1467] Processing CreateTablet for tablet a145fc153ee1427ab611cfc09a05a637 (DEFAULT_TABLE table=test-table [id=d5898e2b0f6945f6a6b835ab9d977812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:58.163733 14421 tablet_service.cc:1467] Processing CreateTablet for tablet a145fc153ee1427ab611cfc09a05a637 (DEFAULT_TABLE table=test-table [id=d5898e2b0f6945f6a6b835ab9d977812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:58.163721 14294 tablet_service.cc:1467] Processing CreateTablet for tablet a145fc153ee1427ab611cfc09a05a637 (DEFAULT_TABLE table=test-table [id=d5898e2b0f6945f6a6b835ab9d977812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:58.164077 14154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a145fc153ee1427ab611cfc09a05a637. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:58.164104 14294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a145fc153ee1427ab611cfc09a05a637. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:58.164104 14421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a145fc153ee1427ab611cfc09a05a637. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:58.167490 14523 tablet_bootstrap.cc:492] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59: Bootstrap starting.
I20260927 18:38:58.167815 14524 tablet_bootstrap.cc:492] T a145fc153ee1427ab611cfc09a05a637 P 0be07e30136e46bba4bd70ed021b5fbe: Bootstrap starting.
I20260927 18:38:58.167825 14525 tablet_bootstrap.cc:492] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6: Bootstrap starting.
I20260927 18:38:58.168375 14525 tablet_bootstrap.cc:654] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:58.168561 14524 tablet_bootstrap.cc:654] T a145fc153ee1427ab611cfc09a05a637 P 0be07e30136e46bba4bd70ed021b5fbe: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:58.168733 14525 log.cc:824] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:58.168751 14523 tablet_bootstrap.cc:654] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:58.168968 14524 log.cc:824] T a145fc153ee1427ab611cfc09a05a637 P 0be07e30136e46bba4bd70ed021b5fbe: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:58.169152 14523 log.cc:824] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:58.169485 14525 tablet_bootstrap.cc:492] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6: No bootstrap required, opened a new log
I20260927 18:38:58.169603 14525 ts_tablet_manager.cc:1397] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:58.169860 14524 tablet_bootstrap.cc:492] T a145fc153ee1427ab611cfc09a05a637 P 0be07e30136e46bba4bd70ed021b5fbe: No bootstrap required, opened a new log
I20260927 18:38:58.169943 14524 ts_tablet_manager.cc:1397] T a145fc153ee1427ab611cfc09a05a637 P 0be07e30136e46bba4bd70ed021b5fbe: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:58.171101 14525 raft_consensus.cc:348] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } } peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } }
I20260927 18:38:58.171254 14525 raft_consensus.cc:374] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:58.171303 14525 raft_consensus.cc:729] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06231b35aef94c21aa17f67a4ce44db6, State: Initialized, Role: FOLLOWER
I20260927 18:38:58.171417 14525 consensus_queue.cc:260] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } } peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } }
I20260927 18:38:58.171708 14360 heartbeater.cc:503] Master 127.11.251.254:39005 was elected leader, sending a full tablet report...
I20260927 18:38:58.171726 14524 raft_consensus.cc:348] T a145fc153ee1427ab611cfc09a05a637 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } } peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } }
I20260927 18:38:58.171898 14524 raft_consensus.cc:374] T a145fc153ee1427ab611cfc09a05a637 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:58.171926 14524 raft_consensus.cc:729] T a145fc153ee1427ab611cfc09a05a637 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0be07e30136e46bba4bd70ed021b5fbe, State: Initialized, Role: FOLLOWER
I20260927 18:38:58.171972 14525 ts_tablet_manager.cc:1428] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:58.172066 14524 consensus_queue.cc:260] T a145fc153ee1427ab611cfc09a05a637 P 0be07e30136e46bba4bd70ed021b5fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } } peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } }
I20260927 18:38:58.172358 14524 ts_tablet_manager.cc:1428] T a145fc153ee1427ab611cfc09a05a637 P 0be07e30136e46bba4bd70ed021b5fbe: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:58.172430 14220 heartbeater.cc:503] Master 127.11.251.254:39005 was elected leader, sending a full tablet report...
I20260927 18:38:58.172807 14523 tablet_bootstrap.cc:492] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59: No bootstrap required, opened a new log
I20260927 18:38:58.172883 14523 ts_tablet_manager.cc:1397] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260927 18:38:58.175005 14523 raft_consensus.cc:348] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } } peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } }
I20260927 18:38:58.175132 14523 raft_consensus.cc:374] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:58.175155 14523 raft_consensus.cc:729] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f4d38fc01f453b9f5369a816fc3c59, State: Initialized, Role: FOLLOWER
I20260927 18:38:58.175293 14523 consensus_queue.cc:260] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } } peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } }
I20260927 18:38:58.175554 14523 ts_tablet_manager.cc:1428] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:38:58.175612 14500 heartbeater.cc:503] Master 127.11.251.254:39005 was elected leader, sending a full tablet report...
I20260927 18:38:58.191612 14531 raft_consensus.cc:482] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:38:58.191745 14531 raft_consensus.cc:504] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } } peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } }
I20260927 18:38:58.192140 14531 leader_election.cc:288] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0be07e30136e46bba4bd70ed021b5fbe (127.11.251.193:39889), 06231b35aef94c21aa17f67a4ce44db6 (127.11.251.194:41337)
I20260927 18:38:58.195420 14522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.192503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:48937 (local address 127.11.251.194:41337)
0927 18:38:58.192564 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:58.192566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:58.192627 (+ 61us) server_negotiation.cc:407] Connection header received
0927 18:38:58.192690 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:58.192691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:58.192702 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:58.192786 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:58.192986 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.193491 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.194681 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.195072 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.195153 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.195164 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:58.195308 (+ 144us) server_negotiation.cc:299] Negotiation successful
0927 18:38:58.195353 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:38:58.195432 14532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.192341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:41337 (local address 127.11.251.195:48937)
0927 18:38:58.192545 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:58.192558 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:58.192623 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:58.192813 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:58.192814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:58.192816 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:58.192908 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.192910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.193619 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.193620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.194591 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.194593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.195161 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.195162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.195236 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.195241 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:38:58.195303 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 18:38:58.195348 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":1,"thread_start_us":111,"threads_started":1}
I20260927 18:38:58.195518 14521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.192381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:35779 (local address 127.11.251.193:39889)
0927 18:38:58.192432 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:58.192435 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:58.192447 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:38:58.192540 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:58.192541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:58.192553 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:58.192635 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:58.192830 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.193500 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.194704 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.195109 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.195210 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.195219 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:58.195394 (+ 175us) server_negotiation.cc:299] Negotiation successful
0927 18:38:58.195433 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 18:38:58.195533 14393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.192320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:39889 (local address 127.11.251.195:35779)
0927 18:38:58.192375 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:58.192390 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:58.192443 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:58.192648 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:58.192649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:58.192651 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:58.192746 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.192748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.193595 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.193596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.194613 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.194615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.195206 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.195207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.195308 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.195311 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:38:58.195375 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 18:38:58.195425 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260927 18:38:58.195789 14314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a145fc153ee1427ab611cfc09a05a637" candidate_uuid: "96f4d38fc01f453b9f5369a816fc3c59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06231b35aef94c21aa17f67a4ce44db6" is_pre_election: true
I20260927 18:38:58.195920 14314 raft_consensus.cc:2454] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96f4d38fc01f453b9f5369a816fc3c59 in term 0.
I20260927 18:38:58.196133 14388 leader_election.cc:302] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06231b35aef94c21aa17f67a4ce44db6, 96f4d38fc01f453b9f5369a816fc3c59; no voters:
I20260927 18:38:58.196173 14174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a145fc153ee1427ab611cfc09a05a637" candidate_uuid: "96f4d38fc01f453b9f5369a816fc3c59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0be07e30136e46bba4bd70ed021b5fbe" is_pre_election: true
I20260927 18:38:58.196285 14531 raft_consensus.cc:2781] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:38:58.196341 14531 raft_consensus.cc:482] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:38:58.196331 14174 raft_consensus.cc:2454] T a145fc153ee1427ab611cfc09a05a637 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96f4d38fc01f453b9f5369a816fc3c59 in term 0.
I20260927 18:38:58.196375 14531 raft_consensus.cc:3037] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:58.197170 14531 raft_consensus.cc:504] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } } peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } }
I20260927 18:38:58.197326 14531 leader_election.cc:288] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 [CANDIDATE]: Term 1 election: Requested vote from peers 0be07e30136e46bba4bd70ed021b5fbe (127.11.251.193:39889), 06231b35aef94c21aa17f67a4ce44db6 (127.11.251.194:41337)
I20260927 18:38:58.197515 14314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a145fc153ee1427ab611cfc09a05a637" candidate_uuid: "96f4d38fc01f453b9f5369a816fc3c59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06231b35aef94c21aa17f67a4ce44db6"
I20260927 18:38:58.197512 14174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a145fc153ee1427ab611cfc09a05a637" candidate_uuid: "96f4d38fc01f453b9f5369a816fc3c59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0be07e30136e46bba4bd70ed021b5fbe"
I20260927 18:38:58.197633 14174 raft_consensus.cc:3037] T a145fc153ee1427ab611cfc09a05a637 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:58.197633 14314 raft_consensus.cc:3037] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:58.198621 14314 raft_consensus.cc:2454] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96f4d38fc01f453b9f5369a816fc3c59 in term 1.
I20260927 18:38:58.198621 14174 raft_consensus.cc:2454] T a145fc153ee1427ab611cfc09a05a637 P 0be07e30136e46bba4bd70ed021b5fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96f4d38fc01f453b9f5369a816fc3c59 in term 1.
I20260927 18:38:58.198803 14388 leader_election.cc:302] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06231b35aef94c21aa17f67a4ce44db6, 96f4d38fc01f453b9f5369a816fc3c59; no voters:
I20260927 18:38:58.198902 14531 raft_consensus.cc:2781] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:58.199079 14531 raft_consensus.cc:686] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 [term 1 LEADER]: Becoming Leader. State: Replica: 96f4d38fc01f453b9f5369a816fc3c59, State: Running, Role: LEADER
I20260927 18:38:58.199179 14531 consensus_queue.cc:237] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } } peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } }
I20260927 18:38:58.200091 14016 catalog_manager.cc:5495] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96f4d38fc01f453b9f5369a816fc3c59 (127.11.251.195). New cstate: current_term: 1 leader_uuid: "96f4d38fc01f453b9f5369a816fc3c59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } health_report { overall_health: UNKNOWN } } }
W20260927 18:38:58.213712 14361 tablet.cc:2369] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 27 18:38:58 dist-test-slave-z47j krb5kdc[13972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534337, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:38:58.233181 14066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.226480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52156 (local address 127.11.251.254:39005)
0927 18:38:58.226553 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:58.226556 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:58.226706 (+ 150us) server_negotiation.cc:407] Connection header received
0927 18:38:58.226777 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:58.226778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:58.226788 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:58.226853 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:58.227313 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.227791 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.228466 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.228640 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.228715 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.230891 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:58.230893 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:58.230893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:58.230899 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:58.232024 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:58.232437 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:58.232438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:58.232438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:58.232486 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:58.232659 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:58.232659 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:58.232659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:58.232751 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:58.232806 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:58.233028 (+ 222us) server_negotiation.cc:299] Negotiation successful
0927 18:38:58.233119 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260927 18:38:58.233407 14540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.226433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:39005 (local address 127.0.0.1:52156)
0927 18:38:58.226623 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:58.226634 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:58.226723 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:58.226858 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:58.226859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:58.227191 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:58.227233 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.227235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.227866 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.227867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.228379 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.228380 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.228716 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.228717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.228769 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.229301 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:58.229305 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:58.230796 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:58.232102 (+ 1306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:58.232108 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:58.232111 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:58.232369 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:58.232546 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:58.232547 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:58.232547 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:58.232604 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:58.232842 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:58.232849 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:58.232919 (+ 70us) client_negotiation.cc:769] Sending connection context
0927 18:38:58.233002 (+ 83us) client_negotiation.cc:240] Negotiation successful
0927 18:38:58.233239 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":80,"thread_start_us":98,"threads_started":1}
I20260927 18:38:58.235499 14016 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:52156:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 18:38:58.235693 14016 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:38:58.237665 14154 tablet_service.cc:1467] Processing CreateTablet for tablet 0a27f38893954aa58ac72c8ec434c7c3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=16d30aaaae4e47b09af52653eafb896c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:38:58.237736 14421 tablet_service.cc:1467] Processing CreateTablet for tablet 0a27f38893954aa58ac72c8ec434c7c3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=16d30aaaae4e47b09af52653eafb896c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:38:58.237854 14154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a27f38893954aa58ac72c8ec434c7c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:58.237854 14421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a27f38893954aa58ac72c8ec434c7c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:58.237784 14294 tablet_service.cc:1467] Processing CreateTablet for tablet 0a27f38893954aa58ac72c8ec434c7c3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=16d30aaaae4e47b09af52653eafb896c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:38:58.238008 14294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a27f38893954aa58ac72c8ec434c7c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:58.239594 14525 tablet_bootstrap.cc:492] T 0a27f38893954aa58ac72c8ec434c7c3 P 06231b35aef94c21aa17f67a4ce44db6: Bootstrap starting.
I20260927 18:38:58.239604 14524 tablet_bootstrap.cc:492] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe: Bootstrap starting.
I20260927 18:38:58.239921 14523 tablet_bootstrap.cc:492] T 0a27f38893954aa58ac72c8ec434c7c3 P 96f4d38fc01f453b9f5369a816fc3c59: Bootstrap starting.
I20260927 18:38:58.240305 14525 tablet_bootstrap.cc:654] T 0a27f38893954aa58ac72c8ec434c7c3 P 06231b35aef94c21aa17f67a4ce44db6: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:58.240530 14523 tablet_bootstrap.cc:654] T 0a27f38893954aa58ac72c8ec434c7c3 P 96f4d38fc01f453b9f5369a816fc3c59: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:58.240530 14524 tablet_bootstrap.cc:654] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:58.240928 14525 tablet_bootstrap.cc:492] T 0a27f38893954aa58ac72c8ec434c7c3 P 06231b35aef94c21aa17f67a4ce44db6: No bootstrap required, opened a new log
I20260927 18:38:58.240962 14525 ts_tablet_manager.cc:1397] T 0a27f38893954aa58ac72c8ec434c7c3 P 06231b35aef94c21aa17f67a4ce44db6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:58.241142 14525 raft_consensus.cc:348] T 0a27f38893954aa58ac72c8ec434c7c3 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:58.241246 14525 raft_consensus.cc:374] T 0a27f38893954aa58ac72c8ec434c7c3 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:58.241272 14525 raft_consensus.cc:729] T 0a27f38893954aa58ac72c8ec434c7c3 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06231b35aef94c21aa17f67a4ce44db6, State: Initialized, Role: FOLLOWER
I20260927 18:38:58.241328 14525 consensus_queue.cc:260] T 0a27f38893954aa58ac72c8ec434c7c3 P 06231b35aef94c21aa17f67a4ce44db6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:58.241384 14524 tablet_bootstrap.cc:492] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe: No bootstrap required, opened a new log
I20260927 18:38:58.241425 14524 ts_tablet_manager.cc:1397] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:58.241428 14525 ts_tablet_manager.cc:1428] T 0a27f38893954aa58ac72c8ec434c7c3 P 06231b35aef94c21aa17f67a4ce44db6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 18:38:58.241609 14524 raft_consensus.cc:348] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:58.241681 14524 raft_consensus.cc:374] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:58.241701 14524 raft_consensus.cc:729] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0be07e30136e46bba4bd70ed021b5fbe, State: Initialized, Role: FOLLOWER
I20260927 18:38:58.241743 14524 consensus_queue.cc:260] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:58.241771 14523 tablet_bootstrap.cc:492] T 0a27f38893954aa58ac72c8ec434c7c3 P 96f4d38fc01f453b9f5369a816fc3c59: No bootstrap required, opened a new log
I20260927 18:38:58.241729 14529 tablet_replica.cc:439] T 0a27f38893954aa58ac72c8ec434c7c3 P 06231b35aef94c21aa17f67a4ce44db6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } } }
I20260927 18:38:58.241818 14523 ts_tablet_manager.cc:1397] T 0a27f38893954aa58ac72c8ec434c7c3 P 96f4d38fc01f453b9f5369a816fc3c59: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:58.241850 14524 ts_tablet_manager.cc:1428] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:58.241854 14529 tablet_replica.cc:442] T 0a27f38893954aa58ac72c8ec434c7c3 P 06231b35aef94c21aa17f67a4ce44db6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:38:58.241878 14530 tablet_replica.cc:439] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } } }
I20260927 18:38:58.241950 14530 tablet_replica.cc:442] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:38:58.242002 14523 raft_consensus.cc:348] T 0a27f38893954aa58ac72c8ec434c7c3 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:58.242085 14523 raft_consensus.cc:374] T 0a27f38893954aa58ac72c8ec434c7c3 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:58.242115 14523 raft_consensus.cc:729] T 0a27f38893954aa58ac72c8ec434c7c3 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f4d38fc01f453b9f5369a816fc3c59, State: Initialized, Role: FOLLOWER
I20260927 18:38:58.242183 14523 consensus_queue.cc:260] T 0a27f38893954aa58ac72c8ec434c7c3 P 96f4d38fc01f453b9f5369a816fc3c59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:58.242293 14523 ts_tablet_manager.cc:1428] T 0a27f38893954aa58ac72c8ec434c7c3 P 96f4d38fc01f453b9f5369a816fc3c59: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260927 18:38:58.242429 14531 tablet_replica.cc:439] T 0a27f38893954aa58ac72c8ec434c7c3 P 96f4d38fc01f453b9f5369a816fc3c59: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } } }
I20260927 18:38:58.242499 14531 tablet_replica.cc:442] T 0a27f38893954aa58ac72c8ec434c7c3 P 96f4d38fc01f453b9f5369a816fc3c59: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:38:58.254045 14530 raft_consensus.cc:482] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:38:58.254129 14530 raft_consensus.cc:504] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:58.254557 14530 leader_election.cc:288] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06231b35aef94c21aa17f67a4ce44db6 (127.11.251.194:41337), 96f4d38fc01f453b9f5369a816fc3c59 (127.11.251.195:38547)
I20260927 18:38:58.257776 14113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.254703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:41337 (local address 127.11.251.193:41879)
0927 18:38:58.254792 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:58.254806 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:58.254851 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:58.255040 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:58.255041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:58.255044 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:58.255144 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.255146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.255836 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.255838 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.256728 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.256730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.257436 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.257437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.257536 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.257541 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:38:58.257648 (+ 107us) client_negotiation.cc:240] Negotiation successful
0927 18:38:58.257703 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":33}
I20260927 18:38:58.257798 14522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.254780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:41879 (local address 127.11.251.194:41337)
0927 18:38:58.254841 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:58.254844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:58.254857 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:38:58.254921 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:58.254922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:58.254932 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:58.255002 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:58.255218 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.255748 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.256845 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.257361 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.257420 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.257428 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:58.257679 (+ 251us) server_negotiation.cc:299] Negotiation successful
0927 18:38:58.257729 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 18:38:58.258083 14520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.254864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:47485 (local address 127.11.251.195:38547)
0927 18:38:58.254923 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:58.254925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:58.255055 (+ 130us) server_negotiation.cc:407] Connection header received
0927 18:38:58.255096 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:58.255097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:58.255108 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:58.255171 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:58.255384 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.255964 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.257026 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.257611 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.257678 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.257687 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:58.257948 (+ 261us) server_negotiation.cc:299] Negotiation successful
0927 18:38:58.258020 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 18:38:58.258083 14314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a27f38893954aa58ac72c8ec434c7c3" candidate_uuid: "0be07e30136e46bba4bd70ed021b5fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06231b35aef94c21aa17f67a4ce44db6" is_pre_election: true
I20260927 18:38:58.258183 14314 raft_consensus.cc:2454] T 0a27f38893954aa58ac72c8ec434c7c3 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0be07e30136e46bba4bd70ed021b5fbe in term 0.
I20260927 18:38:58.258352 14544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.254779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:38547 (local address 127.11.251.193:47485)
0927 18:38:58.254971 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:58.254983 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:58.255047 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:58.255201 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:58.255202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:58.255204 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:58.255279 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.255282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.256051 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.256051 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.256903 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.256905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.257719 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.257721 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.257798 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.257804 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:38:58.258217 (+ 413us) client_negotiation.cc:240] Negotiation successful
0927 18:38:58.258269 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":1,"thread_start_us":112,"threads_started":1}
I20260927 18:38:58.258379 14108 leader_election.cc:302] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06231b35aef94c21aa17f67a4ce44db6, 0be07e30136e46bba4bd70ed021b5fbe; no voters:
I20260927 18:38:58.258540 14530 raft_consensus.cc:2781] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:38:58.258591 14530 raft_consensus.cc:482] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:38:58.258615 14530 raft_consensus.cc:3037] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:58.258725 14454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a27f38893954aa58ac72c8ec434c7c3" candidate_uuid: "0be07e30136e46bba4bd70ed021b5fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f4d38fc01f453b9f5369a816fc3c59" is_pre_election: true
I20260927 18:38:58.258847 14454 raft_consensus.cc:2454] T 0a27f38893954aa58ac72c8ec434c7c3 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0be07e30136e46bba4bd70ed021b5fbe in term 0.
I20260927 18:38:58.259538 14530 raft_consensus.cc:504] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:58.259683 14530 leader_election.cc:288] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe [CANDIDATE]: Term 1 election: Requested vote from peers 06231b35aef94c21aa17f67a4ce44db6 (127.11.251.194:41337), 96f4d38fc01f453b9f5369a816fc3c59 (127.11.251.195:38547)
I20260927 18:38:58.259831 14314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a27f38893954aa58ac72c8ec434c7c3" candidate_uuid: "0be07e30136e46bba4bd70ed021b5fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06231b35aef94c21aa17f67a4ce44db6"
I20260927 18:38:58.259842 14454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a27f38893954aa58ac72c8ec434c7c3" candidate_uuid: "0be07e30136e46bba4bd70ed021b5fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f4d38fc01f453b9f5369a816fc3c59"
I20260927 18:38:58.259907 14454 raft_consensus.cc:3037] T 0a27f38893954aa58ac72c8ec434c7c3 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:58.259907 14314 raft_consensus.cc:3037] T 0a27f38893954aa58ac72c8ec434c7c3 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:58.260737 14454 raft_consensus.cc:2454] T 0a27f38893954aa58ac72c8ec434c7c3 P 96f4d38fc01f453b9f5369a816fc3c59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0be07e30136e46bba4bd70ed021b5fbe in term 1.
I20260927 18:38:58.260737 14314 raft_consensus.cc:2454] T 0a27f38893954aa58ac72c8ec434c7c3 P 06231b35aef94c21aa17f67a4ce44db6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0be07e30136e46bba4bd70ed021b5fbe in term 1.
I20260927 18:38:58.260926 14108 leader_election.cc:302] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06231b35aef94c21aa17f67a4ce44db6, 0be07e30136e46bba4bd70ed021b5fbe; no voters:
I20260927 18:38:58.261083 14530 raft_consensus.cc:2781] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:58.261230 14530 raft_consensus.cc:686] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe [term 1 LEADER]: Becoming Leader. State: Replica: 0be07e30136e46bba4bd70ed021b5fbe, State: Running, Role: LEADER
I20260927 18:38:58.261345 14530 consensus_queue.cc:237] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:58.261868 14545 tablet_replica.cc:439] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe: TxnStatusTablet state changed. Reason: New leader 0be07e30136e46bba4bd70ed021b5fbe. Latest consensus state: current_term: 1 leader_uuid: "0be07e30136e46bba4bd70ed021b5fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } } }
I20260927 18:38:58.261968 14545 tablet_replica.cc:442] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:38:58.262279 14547 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 18:38:58.262223 14016 catalog_manager.cc:5495] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 0be07e30136e46bba4bd70ed021b5fbe (127.11.251.193). New cstate: current_term: 1 leader_uuid: "0be07e30136e46bba4bd70ed021b5fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } health_report { overall_health: UNKNOWN } } }
I20260927 18:38:58.262394 14547 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 18:38:58.273949 14521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.271032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51712 (local address 127.11.251.193:39889)
0927 18:38:58.271096 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:58.271098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:58.271114 (+ 16us) server_negotiation.cc:407] Connection header received
0927 18:38:58.271182 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:58.271183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:58.271199 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:58.271271 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:38:58.271451 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.271933 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.273104 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.273283 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.273344 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.273568 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:38:58.273724 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:38:58.273791 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:58.273882 (+ 91us) server_negotiation.cc:299] Negotiation successful
0927 18:38:58.273901 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:38:58.274012 14540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.270935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:39889 (local address 127.0.0.1:51712)
0927 18:38:58.271031 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:58.271046 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:58.271101 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:58.271294 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:58.271295 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:58.271297 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:38:58.271358 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.271362 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.272014 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.272015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.272995 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.272997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.273372 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.273373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.273447 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.273465 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:38:58.273800 (+ 335us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:38:58.273809 (+ 9us) client_negotiation.cc:769] Sending connection context
0927 18:38:58.273868 (+ 59us) client_negotiation.cc:240] Negotiation successful
0927 18:38:58.273922 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260927 18:38:58.275903 14314 raft_consensus.cc:1261] T 0a27f38893954aa58ac72c8ec434c7c3 P 06231b35aef94c21aa17f67a4ce44db6 [term 1 FOLLOWER]: Refusing update from remote peer 0be07e30136e46bba4bd70ed021b5fbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:58.275938 14454 raft_consensus.cc:1261] T 0a27f38893954aa58ac72c8ec434c7c3 P 96f4d38fc01f453b9f5369a816fc3c59 [term 1 FOLLOWER]: Refusing update from remote peer 0be07e30136e46bba4bd70ed021b5fbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:58.276285 14545 consensus_queue.cc:1035] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:58.276410 14530 consensus_queue.cc:1035] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:58.277664 14531 tablet_replica.cc:439] T 0a27f38893954aa58ac72c8ec434c7c3 P 96f4d38fc01f453b9f5369a816fc3c59: TxnStatusTablet state changed. Reason: New leader 0be07e30136e46bba4bd70ed021b5fbe. Latest consensus state: current_term: 1 leader_uuid: "0be07e30136e46bba4bd70ed021b5fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } } }
I20260927 18:38:58.277773 14531 tablet_replica.cc:442] T 0a27f38893954aa58ac72c8ec434c7c3 P 96f4d38fc01f453b9f5369a816fc3c59: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:38:58.277767 14529 tablet_replica.cc:439] T 0a27f38893954aa58ac72c8ec434c7c3 P 06231b35aef94c21aa17f67a4ce44db6: TxnStatusTablet state changed. Reason: New leader 0be07e30136e46bba4bd70ed021b5fbe. Latest consensus state: current_term: 1 leader_uuid: "0be07e30136e46bba4bd70ed021b5fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } } }
I20260927 18:38:58.277839 14529 tablet_replica.cc:442] T 0a27f38893954aa58ac72c8ec434c7c3 P 06231b35aef94c21aa17f67a4ce44db6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:38:58.278465 14545 tablet_replica.cc:439] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0be07e30136e46bba4bd70ed021b5fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } } }
I20260927 18:38:58.278559 14545 tablet_replica.cc:442] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:38:58.279009 14531 tablet_replica.cc:439] T 0a27f38893954aa58ac72c8ec434c7c3 P 96f4d38fc01f453b9f5369a816fc3c59: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0be07e30136e46bba4bd70ed021b5fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } } }
I20260927 18:38:58.279100 14531 tablet_replica.cc:442] T 0a27f38893954aa58ac72c8ec434c7c3 P 96f4d38fc01f453b9f5369a816fc3c59: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:38:58.279467 14552 mvcc.cc:205] Tried to move back new op lower bound from 7334028649575362560 to 7334028649519333376. Current Snapshot: MvccSnapshot[applied={T|T < 7334028649575362560}]
I20260927 18:38:58.280088 14530 tablet_replica.cc:439] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0be07e30136e46bba4bd70ed021b5fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } } }
I20260927 18:38:58.280198 14530 tablet_replica.cc:442] T 0a27f38893954aa58ac72c8ec434c7c3 P 0be07e30136e46bba4bd70ed021b5fbe: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:38:58.280671 14529 tablet_replica.cc:439] T 0a27f38893954aa58ac72c8ec434c7c3 P 06231b35aef94c21aa17f67a4ce44db6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0be07e30136e46bba4bd70ed021b5fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } } }
I20260927 18:38:58.280776 14529 tablet_replica.cc:442] T 0a27f38893954aa58ac72c8ec434c7c3 P 06231b35aef94c21aa17f67a4ce44db6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:38:58.281416 14551 mvcc.cc:205] Tried to move back new op lower bound from 7334028649575362560 to 7334028649519333376. Current Snapshot: MvccSnapshot[applied={T|T < 7334028649575362560}]
I20260927 18:38:58.283574 14520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.280756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47020 (local address 127.11.251.195:38547)
0927 18:38:58.280797 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:58.280799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:58.280806 (+ 7us) server_negotiation.cc:407] Connection header received
0927 18:38:58.280836 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:58.280837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:58.280855 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:58.280917 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:38:58.281091 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.281873 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.282652 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.282969 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.283046 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.283223 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:38:58.283347 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:38:58.283433 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:58.283479 (+ 46us) server_negotiation.cc:299] Negotiation successful
0927 18:38:58.283503 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:38:58.289131 14506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.286257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:39889 (local address 127.11.251.195:56767)
0927 18:38:58.286340 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:58.286355 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:58.286407 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:58.286542 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:58.286543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:58.286544 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:38:58.286596 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.286597 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.287526 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.287527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.288183 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.288185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.288706 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.288707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.288765 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.288777 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:38:58.288983 (+ 206us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:38:58.288985 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:38:58.289039 (+ 54us) client_negotiation.cc:240] Negotiation successful
0927 18:38:58.289077 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260927 18:38:58.289142 14521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.286333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:56767 (local address 127.11.251.193:39889)
0927 18:38:58.286388 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:58.286390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:58.286401 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:38:58.286461 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:58.286462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:58.286473 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:58.286532 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:38:58.286657 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.287443 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.288283 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.288589 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.288693 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.288871 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:38:58.288907 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:38:58.288966 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:58.289043 (+ 77us) server_negotiation.cc:299] Negotiation successful
0927 18:38:58.289066 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 18:38:58.291738 14314 raft_consensus.cc:1261] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6 [term 1 FOLLOWER]: Refusing update from remote peer 96f4d38fc01f453b9f5369a816fc3c59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:58.291762 14174 raft_consensus.cc:1261] T a145fc153ee1427ab611cfc09a05a637 P 0be07e30136e46bba4bd70ed021b5fbe [term 1 FOLLOWER]: Refusing update from remote peer 96f4d38fc01f453b9f5369a816fc3c59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:58.292035 14533 consensus_queue.cc:1035] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:58.292318 14531 consensus_queue.cc:1035] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:58.294834 14550 mvcc.cc:205] Tried to move back new op lower bound from 7334028649640964096 to 7334028649264709632. Current Snapshot: MvccSnapshot[applied={T|T < 7334028649640964096}]
I20260927 18:38:58.300993 14520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.298126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:47589 (local address 127.11.251.195:38547)
0927 18:38:58.298196 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:58.298199 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:58.298208 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:38:58.298230 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:58.298231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:58.298242 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:58.298298 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:38:58.298441 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.299203 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.300221 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.300521 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.300591 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.300737 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:38:58.300779 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:38:58.300836 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:58.300901 (+ 65us) server_negotiation.cc:299] Negotiation successful
0927 18:38:58.300921 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:38:58.300966 14226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.297933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:38547 (local address 127.11.251.193:47589)
0927 18:38:58.297991 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:58.298011 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:58.298053 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:58.298321 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:58.298322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:58.298323 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:38:58.298377 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.298378 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.299298 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.299299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.300140 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.300143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.300595 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.300596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.300647 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.300659 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:38:58.300843 (+ 184us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:38:58.300845 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:38:58.300883 (+ 38us) client_negotiation.cc:240] Negotiation successful
0927 18:38:58.300918 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260927 18:38:58.310130 14017 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:58.310240 14017 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:58.310935 14017 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545: Sending DeleteTablet for 3 replicas of tablet a145fc153ee1427ab611cfc09a05a637
I20260927 18:38:58.311232 14294 tablet_service.cc:1514] Processing DeleteTablet for tablet a145fc153ee1427ab611cfc09a05a637 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:58 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:33680
I20260927 18:38:58.311275 14154 tablet_service.cc:1514] Processing DeleteTablet for tablet a145fc153ee1427ab611cfc09a05a637 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:58 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:51706
I20260927 18:38:58.311542 14421 tablet_service.cc:1514] Processing DeleteTablet for tablet a145fc153ee1427ab611cfc09a05a637 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:58 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:47012
I20260927 18:38:58.311748 14017 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52154:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:38:58.311901 14017 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:38:58.312986 14154 tablet_service.cc:1467] Processing CreateTablet for tablet 7870fefb40064b9da94eda43cc5ae342 (DEFAULT_TABLE table=test-table [id=012cb72267814a1db457376f06574f80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:58.312986 14294 tablet_service.cc:1467] Processing CreateTablet for tablet 7870fefb40064b9da94eda43cc5ae342 (DEFAULT_TABLE table=test-table [id=012cb72267814a1db457376f06574f80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:58.313110 14154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7870fefb40064b9da94eda43cc5ae342. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:58.313110 14294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7870fefb40064b9da94eda43cc5ae342. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:58.313771 14421 tablet_service.cc:1467] Processing CreateTablet for tablet 7870fefb40064b9da94eda43cc5ae342 (DEFAULT_TABLE table=test-table [id=012cb72267814a1db457376f06574f80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:58.313864 14421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7870fefb40064b9da94eda43cc5ae342. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:58.314479 14524 tablet_bootstrap.cc:492] T 7870fefb40064b9da94eda43cc5ae342 P 0be07e30136e46bba4bd70ed021b5fbe: Bootstrap starting.
I20260927 18:38:58.314929 14524 tablet_bootstrap.cc:654] T 7870fefb40064b9da94eda43cc5ae342 P 0be07e30136e46bba4bd70ed021b5fbe: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:58.315299 14523 tablet_bootstrap.cc:492] T 7870fefb40064b9da94eda43cc5ae342 P 96f4d38fc01f453b9f5369a816fc3c59: Bootstrap starting.
I20260927 18:38:58.315833 14523 tablet_bootstrap.cc:654] T 7870fefb40064b9da94eda43cc5ae342 P 96f4d38fc01f453b9f5369a816fc3c59: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:58.316537 14525 tablet_bootstrap.cc:492] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6: Bootstrap starting.
I20260927 18:38:58.312202 14569 tablet_replica.cc:330] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59: stopping tablet replica
I20260927 18:38:58.316648 14523 tablet_bootstrap.cc:492] T 7870fefb40064b9da94eda43cc5ae342 P 96f4d38fc01f453b9f5369a816fc3c59: No bootstrap required, opened a new log
I20260927 18:38:58.316681 14568 tablet_replica.cc:330] T a145fc153ee1427ab611cfc09a05a637 P 0be07e30136e46bba4bd70ed021b5fbe: stopping tablet replica
I20260927 18:38:58.316695 14523 ts_tablet_manager.cc:1397] T 7870fefb40064b9da94eda43cc5ae342 P 96f4d38fc01f453b9f5369a816fc3c59: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260927 18:38:58.316738 14569 raft_consensus.cc:2229] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:38:58.316817 14568 raft_consensus.cc:2229] T a145fc153ee1427ab611cfc09a05a637 P 0be07e30136e46bba4bd70ed021b5fbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:58.316875 14568 raft_consensus.cc:2258] T a145fc153ee1427ab611cfc09a05a637 P 0be07e30136e46bba4bd70ed021b5fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:58.316847 14523 raft_consensus.cc:348] T 7870fefb40064b9da94eda43cc5ae342 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:58.316936 14523 raft_consensus.cc:374] T 7870fefb40064b9da94eda43cc5ae342 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:58.316962 14523 raft_consensus.cc:729] T 7870fefb40064b9da94eda43cc5ae342 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f4d38fc01f453b9f5369a816fc3c59, State: Initialized, Role: FOLLOWER
I20260927 18:38:58.316955 14525 tablet_bootstrap.cc:654] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:58.317008 14523 consensus_queue.cc:260] T 7870fefb40064b9da94eda43cc5ae342 P 96f4d38fc01f453b9f5369a816fc3c59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:58.317116 14523 ts_tablet_manager.cc:1428] T 7870fefb40064b9da94eda43cc5ae342 P 96f4d38fc01f453b9f5369a816fc3c59: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260927 18:38:58.317260 14524 tablet_bootstrap.cc:492] T 7870fefb40064b9da94eda43cc5ae342 P 0be07e30136e46bba4bd70ed021b5fbe: No bootstrap required, opened a new log
I20260927 18:38:58.317305 14524 ts_tablet_manager.cc:1397] T 7870fefb40064b9da94eda43cc5ae342 P 0be07e30136e46bba4bd70ed021b5fbe: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 18:38:58.316851 14569 raft_consensus.cc:2258] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:58.317446 14524 raft_consensus.cc:348] T 7870fefb40064b9da94eda43cc5ae342 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:58.317526 14524 raft_consensus.cc:374] T 7870fefb40064b9da94eda43cc5ae342 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:58.317545 14524 raft_consensus.cc:729] T 7870fefb40064b9da94eda43cc5ae342 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0be07e30136e46bba4bd70ed021b5fbe, State: Initialized, Role: FOLLOWER
I20260927 18:38:58.317600 14524 consensus_queue.cc:260] T 7870fefb40064b9da94eda43cc5ae342 P 0be07e30136e46bba4bd70ed021b5fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:58.317713 14524 ts_tablet_manager.cc:1428] T 7870fefb40064b9da94eda43cc5ae342 P 0be07e30136e46bba4bd70ed021b5fbe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:58.318097 14569 ts_tablet_manager.cc:1905] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:58.318239 14525 tablet_bootstrap.cc:492] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6: No bootstrap required, opened a new log
I20260927 18:38:58.318289 14525 ts_tablet_manager.cc:1397] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:58.318434 14525 raft_consensus.cc:348] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:58.318501 14525 raft_consensus.cc:374] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:58.318521 14525 raft_consensus.cc:729] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06231b35aef94c21aa17f67a4ce44db6, State: Initialized, Role: FOLLOWER
I20260927 18:38:58.318557 14525 consensus_queue.cc:260] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:58.318562 14568 ts_tablet_manager.cc:1905] T a145fc153ee1427ab611cfc09a05a637 P 0be07e30136e46bba4bd70ed021b5fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:58.318643 14525 ts_tablet_manager.cc:1428] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 18:38:58.320181 14569 ts_tablet_manager.cc:1918] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:38:58.320261 14569 log.cc:1196] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/a145fc153ee1427ab611cfc09a05a637
I20260927 18:38:58.320571 14569 ts_tablet_manager.cc:1939] T a145fc153ee1427ab611cfc09a05a637 P 96f4d38fc01f453b9f5369a816fc3c59: Deleting consensus metadata
I20260927 18:38:58.320998 14002 catalog_manager.cc:4841] TS 96f4d38fc01f453b9f5369a816fc3c59 (127.11.251.195:38547): tablet a145fc153ee1427ab611cfc09a05a637 (table test-table [id=d5898e2b0f6945f6a6b835ab9d977812]) successfully deleted
I20260927 18:38:58.322708 14567 tablet_replica.cc:330] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6: stopping tablet replica
I20260927 18:38:58.322783 14568 ts_tablet_manager.cc:1918] T a145fc153ee1427ab611cfc09a05a637 P 0be07e30136e46bba4bd70ed021b5fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:38:58.322854 14568 log.cc:1196] T a145fc153ee1427ab611cfc09a05a637 P 0be07e30136e46bba4bd70ed021b5fbe: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-0/wal/wals/a145fc153ee1427ab611cfc09a05a637
I20260927 18:38:58.323077 14567 raft_consensus.cc:2229] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:58.323206 14567 raft_consensus.cc:2258] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:58.323450 14568 ts_tablet_manager.cc:1939] T a145fc153ee1427ab611cfc09a05a637 P 0be07e30136e46bba4bd70ed021b5fbe: Deleting consensus metadata
I20260927 18:38:58.323721 14567 ts_tablet_manager.cc:1905] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:58.323861 14001 catalog_manager.cc:4841] TS 0be07e30136e46bba4bd70ed021b5fbe (127.11.251.193:39889): tablet a145fc153ee1427ab611cfc09a05a637 (table test-table [id=d5898e2b0f6945f6a6b835ab9d977812]) successfully deleted
I20260927 18:38:58.324985 14567 ts_tablet_manager.cc:1918] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:38:58.325065 14567 log.cc:1196] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/a145fc153ee1427ab611cfc09a05a637
I20260927 18:38:58.325279 14567 ts_tablet_manager.cc:1939] T a145fc153ee1427ab611cfc09a05a637 P 06231b35aef94c21aa17f67a4ce44db6: Deleting consensus metadata
I20260927 18:38:58.325650 14004 catalog_manager.cc:4841] TS 06231b35aef94c21aa17f67a4ce44db6 (127.11.251.194:41337): tablet a145fc153ee1427ab611cfc09a05a637 (table test-table [id=d5898e2b0f6945f6a6b835ab9d977812]) successfully deleted
I20260927 18:38:58.334860 14529 raft_consensus.cc:482] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:38:58.334939 14529 raft_consensus.cc:504] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:58.335275 14529 leader_election.cc:288] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0be07e30136e46bba4bd70ed021b5fbe (127.11.251.193:39889), 96f4d38fc01f453b9f5369a816fc3c59 (127.11.251.195:38547)
I20260927 18:38:58.339304 14253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.335411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:39889 (local address 127.11.251.194:55041)
0927 18:38:58.335481 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:58.335502 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:58.335580 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:58.335742 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:58.335743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:58.335745 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:58.335832 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.335834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.336848 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.336849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.338154 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.338156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.338982 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.338984 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.339074 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.339078 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:38:58.339127 (+ 49us) client_negotiation.cc:240] Negotiation successful
0927 18:38:58.339186 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 18:38:58.339634 14521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.335542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:55041 (local address 127.11.251.193:39889)
0927 18:38:58.335611 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:58.335613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:58.335624 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:38:58.335651 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:58.335652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:58.335664 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:58.335736 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:58.335931 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.336743 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.338275 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.338865 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.338938 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.338949 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:58.339518 (+ 569us) server_negotiation.cc:299] Negotiation successful
0927 18:38:58.339557 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:38:58.339797 14520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.335495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:58569 (local address 127.11.251.195:38547)
0927 18:38:58.335600 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:58.335603 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:58.335953 (+ 350us) server_negotiation.cc:407] Connection header received
0927 18:38:58.336024 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:58.336025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:58.336035 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:58.336102 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:58.336499 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.337213 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.338669 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.339195 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.339240 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.339248 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:58.339689 (+ 441us) server_negotiation.cc:299] Negotiation successful
0927 18:38:58.339728 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260927 18:38:58.339841 14174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7870fefb40064b9da94eda43cc5ae342" candidate_uuid: "06231b35aef94c21aa17f67a4ce44db6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0be07e30136e46bba4bd70ed021b5fbe" is_pre_election: true
I20260927 18:38:58.339936 14174 raft_consensus.cc:2454] T 7870fefb40064b9da94eda43cc5ae342 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06231b35aef94c21aa17f67a4ce44db6 in term 0.
I20260927 18:38:58.340114 14245 leader_election.cc:302] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06231b35aef94c21aa17f67a4ce44db6, 0be07e30136e46bba4bd70ed021b5fbe; no voters:
I20260927 18:38:58.340266 14529 raft_consensus.cc:2781] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:38:58.340332 14529 raft_consensus.cc:482] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:38:58.340363 14529 raft_consensus.cc:3037] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:58.340467 14573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.335421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:38547 (local address 127.11.251.194:58569)
0927 18:38:58.335898 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:58.335909 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:58.335965 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:58.336107 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:58.336108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:58.336110 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:58.336426 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.336428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.337318 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.337319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.338556 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.338558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.339307 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.339308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.339576 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.339581 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:38:58.340148 (+ 567us) client_negotiation.cc:240] Negotiation successful
0927 18:38:58.340252 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":350,"thread_start_us":67,"threads_started":1}
I20260927 18:38:58.340824 14454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7870fefb40064b9da94eda43cc5ae342" candidate_uuid: "06231b35aef94c21aa17f67a4ce44db6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f4d38fc01f453b9f5369a816fc3c59" is_pre_election: true
I20260927 18:38:58.340910 14454 raft_consensus.cc:2454] T 7870fefb40064b9da94eda43cc5ae342 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06231b35aef94c21aa17f67a4ce44db6 in term 0.
I20260927 18:38:58.341370 14529 raft_consensus.cc:504] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:58.341521 14529 leader_election.cc:288] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 [CANDIDATE]: Term 1 election: Requested vote from peers 0be07e30136e46bba4bd70ed021b5fbe (127.11.251.193:39889), 96f4d38fc01f453b9f5369a816fc3c59 (127.11.251.195:38547)
I20260927 18:38:58.341758 14454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7870fefb40064b9da94eda43cc5ae342" candidate_uuid: "06231b35aef94c21aa17f67a4ce44db6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f4d38fc01f453b9f5369a816fc3c59"
I20260927 18:38:58.341827 14454 raft_consensus.cc:3037] T 7870fefb40064b9da94eda43cc5ae342 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:58.341857 14174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7870fefb40064b9da94eda43cc5ae342" candidate_uuid: "06231b35aef94c21aa17f67a4ce44db6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0be07e30136e46bba4bd70ed021b5fbe"
I20260927 18:38:58.341929 14174 raft_consensus.cc:3037] T 7870fefb40064b9da94eda43cc5ae342 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:58.342711 14454 raft_consensus.cc:2454] T 7870fefb40064b9da94eda43cc5ae342 P 96f4d38fc01f453b9f5369a816fc3c59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06231b35aef94c21aa17f67a4ce44db6 in term 1.
I20260927 18:38:58.342809 14174 raft_consensus.cc:2454] T 7870fefb40064b9da94eda43cc5ae342 P 0be07e30136e46bba4bd70ed021b5fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06231b35aef94c21aa17f67a4ce44db6 in term 1.
I20260927 18:38:58.343034 14246 leader_election.cc:302] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06231b35aef94c21aa17f67a4ce44db6, 96f4d38fc01f453b9f5369a816fc3c59; no voters:
I20260927 18:38:58.343178 14529 raft_consensus.cc:2781] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:58.343309 14529 raft_consensus.cc:686] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 [term 1 LEADER]: Becoming Leader. State: Replica: 06231b35aef94c21aa17f67a4ce44db6, State: Running, Role: LEADER
I20260927 18:38:58.343387 14529 consensus_queue.cc:237] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:58.344112 14015 catalog_manager.cc:5495] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06231b35aef94c21aa17f67a4ce44db6 (127.11.251.194). New cstate: current_term: 1 leader_uuid: "06231b35aef94c21aa17f67a4ce44db6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } health_report { overall_health: UNKNOWN } } }
I20260927 18:38:58.358809 14522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.355421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33692 (local address 127.11.251.194:41337)
0927 18:38:58.355496 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:58.355499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:58.355509 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:38:58.355553 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:58.355554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:58.355578 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:58.355667 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:38:58.355832 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.356649 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.357756 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.358065 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.358153 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.358350 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:38:58.358512 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:38:58.358606 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:58.358689 (+ 83us) server_negotiation.cc:299] Negotiation successful
0927 18:38:58.358716 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 18:38:58.365438 14521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.362257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:37023 (local address 127.11.251.193:39889)
0927 18:38:58.362300 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:58.362302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:58.362353 (+ 51us) server_negotiation.cc:407] Connection header received
0927 18:38:58.362445 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:58.362446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:58.362457 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:58.362516 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:38:58.362694 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.363394 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.364563 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.364856 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.364939 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.365155 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:38:58.365192 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:38:58.365257 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:58.365343 (+ 86us) server_negotiation.cc:299] Negotiation successful
0927 18:38:58.365362 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:38:58.365486 14366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.362182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:39889 (local address 127.11.251.194:37023)
0927 18:38:58.362291 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:58.362304 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:58.362363 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:58.362528 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:58.362528 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:58.362530 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:38:58.362601 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.362603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.363491 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.363492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.364448 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.364450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.364951 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.364952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.365027 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.365045 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:38:58.365272 (+ 227us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:38:58.365274 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:38:58.365337 (+ 63us) client_negotiation.cc:240] Negotiation successful
0927 18:38:58.365391 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260927 18:38:58.368947 14174 raft_consensus.cc:1261] T 7870fefb40064b9da94eda43cc5ae342 P 0be07e30136e46bba4bd70ed021b5fbe [term 1 FOLLOWER]: Refusing update from remote peer 06231b35aef94c21aa17f67a4ce44db6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:58.369378 14454 raft_consensus.cc:1261] T 7870fefb40064b9da94eda43cc5ae342 P 96f4d38fc01f453b9f5369a816fc3c59 [term 1 FOLLOWER]: Refusing update from remote peer 06231b35aef94c21aa17f67a4ce44db6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:58.369367 14529 consensus_queue.cc:1035] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:58.369673 14529 consensus_queue.cc:1035] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:58.382197 14226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.378807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:41337 (local address 127.11.251.193:40593)
0927 18:38:58.378944 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:58.378957 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:58.379002 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:58.379219 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:58.379220 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:58.379221 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:38:58.379266 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.379268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.380187 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.380189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.381143 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:58.381146 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.381744 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.381745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:58.381810 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.381816 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:38:58.382000 (+ 184us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:38:58.382001 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 18:38:58.382042 (+ 41us) client_negotiation.cc:240] Negotiation successful
0927 18:38:58.382104 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260927 18:38:58.382475 14522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:58.378947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:40593 (local address 127.11.251.194:41337)
0927 18:38:58.378997 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:58.378999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:58.379011 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:38:58.379093 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:58.379093 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:58.379103 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:58.379159 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:38:58.379343 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.380082 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.381263 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:58.381561 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:58.381716 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:58.381896 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:38:58.381932 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:38:58.381978 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:58.382387 (+ 409us) server_negotiation.cc:299] Negotiation successful
0927 18:38:58.382409 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260927 18:38:58.382714 14551 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260927 18:38:59.908561 14015 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:59.908706 14015 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:59.909741 14015 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545: Sending DeleteTablet for 3 replicas of tablet 7870fefb40064b9da94eda43cc5ae342
I20260927 18:38:59.910041 14294 tablet_service.cc:1514] Processing DeleteTablet for tablet 7870fefb40064b9da94eda43cc5ae342 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:59 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:33680
I20260927 18:38:59.909998 14154 tablet_service.cc:1514] Processing DeleteTablet for tablet 7870fefb40064b9da94eda43cc5ae342 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:59 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:51706
I20260927 18:38:59.910269 14421 tablet_service.cc:1514] Processing DeleteTablet for tablet 7870fefb40064b9da94eda43cc5ae342 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:59 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:47012
I20260927 18:38:59.913695 14605 tablet_replica.cc:330] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6: stopping tablet replica
I20260927 18:38:59.913791 14605 raft_consensus.cc:2229] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:38:59.913856 14604 tablet_replica.cc:330] T 7870fefb40064b9da94eda43cc5ae342 P 0be07e30136e46bba4bd70ed021b5fbe: stopping tablet replica
I20260927 18:38:59.913921 14604 raft_consensus.cc:2229] T 7870fefb40064b9da94eda43cc5ae342 P 0be07e30136e46bba4bd70ed021b5fbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:59.913954 14606 tablet_replica.cc:330] T 7870fefb40064b9da94eda43cc5ae342 P 96f4d38fc01f453b9f5369a816fc3c59: stopping tablet replica
I20260927 18:38:59.913990 14604 raft_consensus.cc:2258] T 7870fefb40064b9da94eda43cc5ae342 P 0be07e30136e46bba4bd70ed021b5fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:59.914129 14606 raft_consensus.cc:2229] T 7870fefb40064b9da94eda43cc5ae342 P 96f4d38fc01f453b9f5369a816fc3c59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:59.914004 14605 raft_consensus.cc:2258] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:59.914233 14604 ts_tablet_manager.cc:1905] T 7870fefb40064b9da94eda43cc5ae342 P 0be07e30136e46bba4bd70ed021b5fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:59.914393 14605 ts_tablet_manager.cc:1905] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:59.914397 14606 raft_consensus.cc:2258] T 7870fefb40064b9da94eda43cc5ae342 P 96f4d38fc01f453b9f5369a816fc3c59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:59.914767 14606 ts_tablet_manager.cc:1905] T 7870fefb40064b9da94eda43cc5ae342 P 96f4d38fc01f453b9f5369a816fc3c59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:59.916110 14604 ts_tablet_manager.cc:1918] T 7870fefb40064b9da94eda43cc5ae342 P 0be07e30136e46bba4bd70ed021b5fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 18:38:59.916229 14604 log.cc:1196] T 7870fefb40064b9da94eda43cc5ae342 P 0be07e30136e46bba4bd70ed021b5fbe: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-0/wal/wals/7870fefb40064b9da94eda43cc5ae342
I20260927 18:38:59.916337 14605 ts_tablet_manager.cc:1918] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 18:38:59.916512 14604 ts_tablet_manager.cc:1939] T 7870fefb40064b9da94eda43cc5ae342 P 0be07e30136e46bba4bd70ed021b5fbe: Deleting consensus metadata
I20260927 18:38:59.916543 14605 log.cc:1196] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/7870fefb40064b9da94eda43cc5ae342
I20260927 18:38:59.916858 14001 catalog_manager.cc:4841] TS 0be07e30136e46bba4bd70ed021b5fbe (127.11.251.193:39889): tablet 7870fefb40064b9da94eda43cc5ae342 (table test-table [id=012cb72267814a1db457376f06574f80]) successfully deleted
I20260927 18:38:59.916955 14605 ts_tablet_manager.cc:1939] T 7870fefb40064b9da94eda43cc5ae342 P 06231b35aef94c21aa17f67a4ce44db6: Deleting consensus metadata
I20260927 18:38:59.917303 14606 ts_tablet_manager.cc:1918] T 7870fefb40064b9da94eda43cc5ae342 P 96f4d38fc01f453b9f5369a816fc3c59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 18:38:59.917290 14004 catalog_manager.cc:4841] TS 06231b35aef94c21aa17f67a4ce44db6 (127.11.251.194:41337): tablet 7870fefb40064b9da94eda43cc5ae342 (table test-table [id=012cb72267814a1db457376f06574f80]) successfully deleted
I20260927 18:38:59.917430 14606 log.cc:1196] T 7870fefb40064b9da94eda43cc5ae342 P 96f4d38fc01f453b9f5369a816fc3c59: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/7870fefb40064b9da94eda43cc5ae342
I20260927 18:38:59.917744 14606 ts_tablet_manager.cc:1939] T 7870fefb40064b9da94eda43cc5ae342 P 96f4d38fc01f453b9f5369a816fc3c59: Deleting consensus metadata
I20260927 18:38:59.918160 14002 catalog_manager.cc:4841] TS 96f4d38fc01f453b9f5369a816fc3c59 (127.11.251.195:38547): tablet 7870fefb40064b9da94eda43cc5ae342 (table test-table [id=012cb72267814a1db457376f06574f80]) successfully deleted
Sep 27 18:38:59 dist-test-slave-z47j krb5kdc[13972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534339, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 18:38:59 dist-test-slave-z47j krb5kdc[13972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534339, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:38:59.936085 14614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:59.928182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52168 (local address 127.11.251.254:39005)
0927 18:38:59.928330 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:59.928332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:59.928372 (+ 40us) server_negotiation.cc:407] Connection header received
0927 18:38:59.928447 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:59.928447 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:59.928458 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:59.928533 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:59.929142 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:59.929758 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:59.930555 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:59.930778 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:59.930842 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:59.933313 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:59.933314 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:59.933315 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:59.933322 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:59.934979 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:59.935326 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:59.935327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:59.935328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:59.935387 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:59.935594 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:59.935595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:59.935595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:59.935710 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:59.935797 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:59.935921 (+ 124us) server_negotiation.cc:299] Negotiation successful
0927 18:38:59.935981 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":92,"threads_started":1}
I20260927 18:38:59.937952 14015 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:38:59.938145 14015 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:38:59.939822 14294 tablet_service.cc:1467] Processing CreateTablet for tablet 46ffa141183a41b79185c5ad7645cf47 (DEFAULT_TABLE table=test-table [id=5078fe2098144939a1addba045564062]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:59.939857 14421 tablet_service.cc:1467] Processing CreateTablet for tablet 46ffa141183a41b79185c5ad7645cf47 (DEFAULT_TABLE table=test-table [id=5078fe2098144939a1addba045564062]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:59.939986 14294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46ffa141183a41b79185c5ad7645cf47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:59.940021 14421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46ffa141183a41b79185c5ad7645cf47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:59.941188 14615 tablet_bootstrap.cc:492] T 46ffa141183a41b79185c5ad7645cf47 P 06231b35aef94c21aa17f67a4ce44db6: Bootstrap starting.
I20260927 18:38:59.941236 14154 tablet_service.cc:1467] Processing CreateTablet for tablet 46ffa141183a41b79185c5ad7645cf47 (DEFAULT_TABLE table=test-table [id=5078fe2098144939a1addba045564062]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:59.941372 14154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46ffa141183a41b79185c5ad7645cf47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:59.941741 14615 tablet_bootstrap.cc:654] T 46ffa141183a41b79185c5ad7645cf47 P 06231b35aef94c21aa17f67a4ce44db6: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:59.942083 14616 tablet_bootstrap.cc:492] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59: Bootstrap starting.
I20260927 18:38:59.942365 14615 tablet_bootstrap.cc:492] T 46ffa141183a41b79185c5ad7645cf47 P 06231b35aef94c21aa17f67a4ce44db6: No bootstrap required, opened a new log
I20260927 18:38:59.942438 14615 ts_tablet_manager.cc:1397] T 46ffa141183a41b79185c5ad7645cf47 P 06231b35aef94c21aa17f67a4ce44db6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260927 18:38:59.942739 14618 tablet_bootstrap.cc:492] T 46ffa141183a41b79185c5ad7645cf47 P 0be07e30136e46bba4bd70ed021b5fbe: Bootstrap starting.
I20260927 18:38:59.942584 14615 raft_consensus.cc:348] T 46ffa141183a41b79185c5ad7645cf47 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:59.942791 14615 raft_consensus.cc:374] T 46ffa141183a41b79185c5ad7645cf47 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:59.942850 14615 raft_consensus.cc:729] T 46ffa141183a41b79185c5ad7645cf47 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06231b35aef94c21aa17f67a4ce44db6, State: Initialized, Role: FOLLOWER
I20260927 18:38:59.942951 14615 consensus_queue.cc:260] T 46ffa141183a41b79185c5ad7645cf47 P 06231b35aef94c21aa17f67a4ce44db6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:59.943113 14615 ts_tablet_manager.cc:1428] T 46ffa141183a41b79185c5ad7645cf47 P 06231b35aef94c21aa17f67a4ce44db6: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260927 18:38:59.943199 14618 tablet_bootstrap.cc:654] T 46ffa141183a41b79185c5ad7645cf47 P 0be07e30136e46bba4bd70ed021b5fbe: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:59.943743 14616 tablet_bootstrap.cc:654] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:59.944110 14618 tablet_bootstrap.cc:492] T 46ffa141183a41b79185c5ad7645cf47 P 0be07e30136e46bba4bd70ed021b5fbe: No bootstrap required, opened a new log
I20260927 18:38:59.944176 14618 ts_tablet_manager.cc:1397] T 46ffa141183a41b79185c5ad7645cf47 P 0be07e30136e46bba4bd70ed021b5fbe: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:59.944300 14618 raft_consensus.cc:348] T 46ffa141183a41b79185c5ad7645cf47 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:59.944372 14618 raft_consensus.cc:374] T 46ffa141183a41b79185c5ad7645cf47 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:59.944387 14618 raft_consensus.cc:729] T 46ffa141183a41b79185c5ad7645cf47 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0be07e30136e46bba4bd70ed021b5fbe, State: Initialized, Role: FOLLOWER
I20260927 18:38:59.944453 14618 consensus_queue.cc:260] T 46ffa141183a41b79185c5ad7645cf47 P 0be07e30136e46bba4bd70ed021b5fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:59.944584 14618 ts_tablet_manager.cc:1428] T 46ffa141183a41b79185c5ad7645cf47 P 0be07e30136e46bba4bd70ed021b5fbe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:59.944640 14616 tablet_bootstrap.cc:492] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59: No bootstrap required, opened a new log
I20260927 18:38:59.944690 14616 ts_tablet_manager.cc:1397] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 18:38:59.944813 14616 raft_consensus.cc:348] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:59.944897 14616 raft_consensus.cc:374] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:59.944922 14616 raft_consensus.cc:729] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f4d38fc01f453b9f5369a816fc3c59, State: Initialized, Role: FOLLOWER
I20260927 18:38:59.944995 14616 consensus_queue.cc:260] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:38:59.945175 14616 ts_tablet_manager.cc:1428] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20260927 18:39:00.054651 14221 tablet.cc:2369] T 46ffa141183a41b79185c5ad7645cf47 P 0be07e30136e46bba4bd70ed021b5fbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:39:00.062573 14621 raft_consensus.cc:482] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:39:00.062708 14621 raft_consensus.cc:504] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:39:00.062951 14621 leader_election.cc:288] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0be07e30136e46bba4bd70ed021b5fbe (127.11.251.193:39889), 06231b35aef94c21aa17f67a4ce44db6 (127.11.251.194:41337)
I20260927 18:39:00.063266 14314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46ffa141183a41b79185c5ad7645cf47" candidate_uuid: "96f4d38fc01f453b9f5369a816fc3c59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06231b35aef94c21aa17f67a4ce44db6" is_pre_election: true
I20260927 18:39:00.063265 14174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46ffa141183a41b79185c5ad7645cf47" candidate_uuid: "96f4d38fc01f453b9f5369a816fc3c59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0be07e30136e46bba4bd70ed021b5fbe" is_pre_election: true
I20260927 18:39:00.063419 14314 raft_consensus.cc:2454] T 46ffa141183a41b79185c5ad7645cf47 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96f4d38fc01f453b9f5369a816fc3c59 in term 0.
I20260927 18:39:00.063419 14174 raft_consensus.cc:2454] T 46ffa141183a41b79185c5ad7645cf47 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96f4d38fc01f453b9f5369a816fc3c59 in term 0.
I20260927 18:39:00.063633 14388 leader_election.cc:302] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06231b35aef94c21aa17f67a4ce44db6, 96f4d38fc01f453b9f5369a816fc3c59; no voters:
I20260927 18:39:00.063738 14621 raft_consensus.cc:2781] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:39:00.063784 14621 raft_consensus.cc:482] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:39:00.063804 14621 raft_consensus.cc:3037] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:00.064653 14621 raft_consensus.cc:504] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:39:00.064788 14621 leader_election.cc:288] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 [CANDIDATE]: Term 1 election: Requested vote from peers 0be07e30136e46bba4bd70ed021b5fbe (127.11.251.193:39889), 06231b35aef94c21aa17f67a4ce44db6 (127.11.251.194:41337)
I20260927 18:39:00.064975 14314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46ffa141183a41b79185c5ad7645cf47" candidate_uuid: "96f4d38fc01f453b9f5369a816fc3c59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06231b35aef94c21aa17f67a4ce44db6"
I20260927 18:39:00.065045 14314 raft_consensus.cc:3037] T 46ffa141183a41b79185c5ad7645cf47 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:00.065076 14174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46ffa141183a41b79185c5ad7645cf47" candidate_uuid: "96f4d38fc01f453b9f5369a816fc3c59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0be07e30136e46bba4bd70ed021b5fbe"
I20260927 18:39:00.065238 14174 raft_consensus.cc:3037] T 46ffa141183a41b79185c5ad7645cf47 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:00.065924 14314 raft_consensus.cc:2454] T 46ffa141183a41b79185c5ad7645cf47 P 06231b35aef94c21aa17f67a4ce44db6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96f4d38fc01f453b9f5369a816fc3c59 in term 1.
I20260927 18:39:00.066120 14388 leader_election.cc:302] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06231b35aef94c21aa17f67a4ce44db6, 96f4d38fc01f453b9f5369a816fc3c59; no voters:
I20260927 18:39:00.066210 14621 raft_consensus.cc:2781] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:00.066231 14174 raft_consensus.cc:2454] T 46ffa141183a41b79185c5ad7645cf47 P 0be07e30136e46bba4bd70ed021b5fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96f4d38fc01f453b9f5369a816fc3c59 in term 1.
I20260927 18:39:00.066439 14621 raft_consensus.cc:686] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 [term 1 LEADER]: Becoming Leader. State: Replica: 96f4d38fc01f453b9f5369a816fc3c59, State: Running, Role: LEADER
I20260927 18:39:00.066520 14621 consensus_queue.cc:237] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:39:00.067361 14017 catalog_manager.cc:5495] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96f4d38fc01f453b9f5369a816fc3c59 (127.11.251.195). New cstate: current_term: 1 leader_uuid: "96f4d38fc01f453b9f5369a816fc3c59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } health_report { overall_health: HEALTHY } } }
I20260927 18:39:00.093535 14625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:00.090990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47024 (local address 127.11.251.195:38547)
0927 18:39:00.091167 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:00.091168 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:00.091174 (+ 6us) server_negotiation.cc:407] Connection header received
0927 18:39:00.091194 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:00.091194 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:00.091205 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:00.091288 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:39:00.091480 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:00.092020 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:00.092798 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:00.093023 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:00.093096 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:00.093241 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:39:00.093277 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:39:00.093342 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:00.093404 (+ 62us) server_negotiation.cc:299] Negotiation successful
0927 18:39:00.093427 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260927 18:39:00.096204 14314 raft_consensus.cc:1261] T 46ffa141183a41b79185c5ad7645cf47 P 06231b35aef94c21aa17f67a4ce44db6 [term 1 FOLLOWER]: Refusing update from remote peer 96f4d38fc01f453b9f5369a816fc3c59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:00.096217 14174 raft_consensus.cc:1261] T 46ffa141183a41b79185c5ad7645cf47 P 0be07e30136e46bba4bd70ed021b5fbe [term 1 FOLLOWER]: Refusing update from remote peer 96f4d38fc01f453b9f5369a816fc3c59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:00.096414 14622 consensus_queue.cc:1035] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:00.096529 14621 consensus_queue.cc:1035] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:00.107996 14017 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:00.108119 14017 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:00.108857 14017 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545: Sending DeleteTablet for 3 replicas of tablet 46ffa141183a41b79185c5ad7645cf47
I20260927 18:39:00.109160 14154 tablet_service.cc:1514] Processing DeleteTablet for tablet 46ffa141183a41b79185c5ad7645cf47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:00 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:51706
I20260927 18:39:00.109203 14421 tablet_service.cc:1514] Processing DeleteTablet for tablet 46ffa141183a41b79185c5ad7645cf47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:00 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:47012
I20260927 18:39:00.109293 14294 tablet_service.cc:1514] Processing DeleteTablet for tablet 46ffa141183a41b79185c5ad7645cf47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:00 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:33680
I20260927 18:39:00.109385 14604 tablet_replica.cc:330] T 46ffa141183a41b79185c5ad7645cf47 P 0be07e30136e46bba4bd70ed021b5fbe: stopping tablet replica
I20260927 18:39:00.109443 14604 raft_consensus.cc:2229] T 46ffa141183a41b79185c5ad7645cf47 P 0be07e30136e46bba4bd70ed021b5fbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:00.109488 14604 raft_consensus.cc:2258] T 46ffa141183a41b79185c5ad7645cf47 P 0be07e30136e46bba4bd70ed021b5fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:00.109629 14605 tablet_replica.cc:330] T 46ffa141183a41b79185c5ad7645cf47 P 06231b35aef94c21aa17f67a4ce44db6: stopping tablet replica
I20260927 18:39:00.109690 14605 raft_consensus.cc:2229] T 46ffa141183a41b79185c5ad7645cf47 P 06231b35aef94c21aa17f67a4ce44db6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:00.109728 14605 raft_consensus.cc:2258] T 46ffa141183a41b79185c5ad7645cf47 P 06231b35aef94c21aa17f67a4ce44db6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:00.109663 14017 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 18:39:00.109776 14606 tablet_replica.cc:330] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59: stopping tablet replica
I20260927 18:39:00.109835 14606 raft_consensus.cc:2229] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 [term 1 LEADER]: Raft consensus shutting down.
W20260927 18:39:00.109839 14017 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:39:00.109892 14606 raft_consensus.cc:2258] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:00.109903 14604 ts_tablet_manager.cc:1905] T 46ffa141183a41b79185c5ad7645cf47 P 0be07e30136e46bba4bd70ed021b5fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:00.110379 14606 ts_tablet_manager.cc:1905] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:00.111143 14421 tablet_service.cc:1467] Processing CreateTablet for tablet 39fb2af24c9446068cb639312bd49dd9 (DEFAULT_TABLE table=test-table [id=deb47fb249e048ddb7171dbeb4aa0f09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:00.111174 14154 tablet_service.cc:1467] Processing CreateTablet for tablet 39fb2af24c9446068cb639312bd49dd9 (DEFAULT_TABLE table=test-table [id=deb47fb249e048ddb7171dbeb4aa0f09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:00.111289 14154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39fb2af24c9446068cb639312bd49dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:00.111296 14421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39fb2af24c9446068cb639312bd49dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:00.112078 14604 ts_tablet_manager.cc:1918] T 46ffa141183a41b79185c5ad7645cf47 P 0be07e30136e46bba4bd70ed021b5fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:39:00.112139 14604 log.cc:1196] T 46ffa141183a41b79185c5ad7645cf47 P 0be07e30136e46bba4bd70ed021b5fbe: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-0/wal/wals/46ffa141183a41b79185c5ad7645cf47
I20260927 18:39:00.112308 14606 ts_tablet_manager.cc:1918] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:39:00.112365 14606 log.cc:1196] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/46ffa141183a41b79185c5ad7645cf47
I20260927 18:39:00.112425 14604 ts_tablet_manager.cc:1939] T 46ffa141183a41b79185c5ad7645cf47 P 0be07e30136e46bba4bd70ed021b5fbe: Deleting consensus metadata
I20260927 18:39:00.112635 14606 ts_tablet_manager.cc:1939] T 46ffa141183a41b79185c5ad7645cf47 P 96f4d38fc01f453b9f5369a816fc3c59: Deleting consensus metadata
I20260927 18:39:00.112942 14618 tablet_bootstrap.cc:492] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe: Bootstrap starting.
I20260927 18:39:00.113021 14616 tablet_bootstrap.cc:492] T 39fb2af24c9446068cb639312bd49dd9 P 96f4d38fc01f453b9f5369a816fc3c59: Bootstrap starting.
I20260927 18:39:00.113024 14001 catalog_manager.cc:4841] TS 0be07e30136e46bba4bd70ed021b5fbe (127.11.251.193:39889): tablet 46ffa141183a41b79185c5ad7645cf47 (table test-table [id=5078fe2098144939a1addba045564062]) successfully deleted
I20260927 18:39:00.113093 14605 ts_tablet_manager.cc:1905] T 46ffa141183a41b79185c5ad7645cf47 P 06231b35aef94c21aa17f67a4ce44db6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:00.113373 14618 tablet_bootstrap.cc:654] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:00.113363 14616 tablet_bootstrap.cc:654] T 39fb2af24c9446068cb639312bd49dd9 P 96f4d38fc01f453b9f5369a816fc3c59: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:00.113476 14294 tablet_service.cc:1467] Processing CreateTablet for tablet 39fb2af24c9446068cb639312bd49dd9 (DEFAULT_TABLE table=test-table [id=deb47fb249e048ddb7171dbeb4aa0f09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:00.113603 14294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39fb2af24c9446068cb639312bd49dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:00.114001 14002 catalog_manager.cc:4841] TS 96f4d38fc01f453b9f5369a816fc3c59 (127.11.251.195:38547): tablet 46ffa141183a41b79185c5ad7645cf47 (table test-table [id=5078fe2098144939a1addba045564062]) successfully deleted
I20260927 18:39:00.114280 14616 tablet_bootstrap.cc:492] T 39fb2af24c9446068cb639312bd49dd9 P 96f4d38fc01f453b9f5369a816fc3c59: No bootstrap required, opened a new log
I20260927 18:39:00.114280 14618 tablet_bootstrap.cc:492] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe: No bootstrap required, opened a new log
I20260927 18:39:00.114331 14618 ts_tablet_manager.cc:1397] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260927 18:39:00.114331 14616 ts_tablet_manager.cc:1397] T 39fb2af24c9446068cb639312bd49dd9 P 96f4d38fc01f453b9f5369a816fc3c59: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260927 18:39:00.114467 14616 raft_consensus.cc:348] T 39fb2af24c9446068cb639312bd49dd9 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:39:00.114467 14618 raft_consensus.cc:348] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:39:00.114532 14616 raft_consensus.cc:374] T 39fb2af24c9446068cb639312bd49dd9 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:00.114550 14618 raft_consensus.cc:374] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:00.114557 14616 raft_consensus.cc:729] T 39fb2af24c9446068cb639312bd49dd9 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f4d38fc01f453b9f5369a816fc3c59, State: Initialized, Role: FOLLOWER
I20260927 18:39:00.114574 14618 raft_consensus.cc:729] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0be07e30136e46bba4bd70ed021b5fbe, State: Initialized, Role: FOLLOWER
I20260927 18:39:00.114609 14616 consensus_queue.cc:260] T 39fb2af24c9446068cb639312bd49dd9 P 96f4d38fc01f453b9f5369a816fc3c59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:39:00.114619 14618 consensus_queue.cc:260] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:39:00.114656 14605 ts_tablet_manager.cc:1918] T 46ffa141183a41b79185c5ad7645cf47 P 06231b35aef94c21aa17f67a4ce44db6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:39:00.114708 14605 log.cc:1196] T 46ffa141183a41b79185c5ad7645cf47 P 06231b35aef94c21aa17f67a4ce44db6: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/46ffa141183a41b79185c5ad7645cf47
I20260927 18:39:00.114709 14616 ts_tablet_manager.cc:1428] T 39fb2af24c9446068cb639312bd49dd9 P 96f4d38fc01f453b9f5369a816fc3c59: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:00.114709 14618 ts_tablet_manager.cc:1428] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:00.114776 14615 tablet_bootstrap.cc:492] T 39fb2af24c9446068cb639312bd49dd9 P 06231b35aef94c21aa17f67a4ce44db6: Bootstrap starting.
I20260927 18:39:00.114924 14605 ts_tablet_manager.cc:1939] T 46ffa141183a41b79185c5ad7645cf47 P 06231b35aef94c21aa17f67a4ce44db6: Deleting consensus metadata
I20260927 18:39:00.115075 14615 tablet_bootstrap.cc:654] T 39fb2af24c9446068cb639312bd49dd9 P 06231b35aef94c21aa17f67a4ce44db6: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:00.115728 14615 tablet_bootstrap.cc:492] T 39fb2af24c9446068cb639312bd49dd9 P 06231b35aef94c21aa17f67a4ce44db6: No bootstrap required, opened a new log
I20260927 18:39:00.115777 14615 ts_tablet_manager.cc:1397] T 39fb2af24c9446068cb639312bd49dd9 P 06231b35aef94c21aa17f67a4ce44db6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260927 18:39:00.115885 14615 raft_consensus.cc:348] T 39fb2af24c9446068cb639312bd49dd9 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:39:00.115952 14615 raft_consensus.cc:374] T 39fb2af24c9446068cb639312bd49dd9 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:00.115975 14615 raft_consensus.cc:729] T 39fb2af24c9446068cb639312bd49dd9 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06231b35aef94c21aa17f67a4ce44db6, State: Initialized, Role: FOLLOWER
I20260927 18:39:00.116020 14615 consensus_queue.cc:260] T 39fb2af24c9446068cb639312bd49dd9 P 06231b35aef94c21aa17f67a4ce44db6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:39:00.116128 14615 ts_tablet_manager.cc:1428] T 39fb2af24c9446068cb639312bd49dd9 P 06231b35aef94c21aa17f67a4ce44db6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:00.116159 14004 catalog_manager.cc:4841] TS 06231b35aef94c21aa17f67a4ce44db6 (127.11.251.194:41337): tablet 46ffa141183a41b79185c5ad7645cf47 (table test-table [id=5078fe2098144939a1addba045564062]) successfully deleted
W20260927 18:39:00.124653 14501 tablet.cc:2369] T 39fb2af24c9446068cb639312bd49dd9 P 96f4d38fc01f453b9f5369a816fc3c59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:39:00.258599 14591 raft_consensus.cc:482] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:39:00.258771 14591 raft_consensus.cc:504] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:39:00.259018 14591 leader_election.cc:288] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06231b35aef94c21aa17f67a4ce44db6 (127.11.251.194:41337), 96f4d38fc01f453b9f5369a816fc3c59 (127.11.251.195:38547)
I20260927 18:39:00.259317 14314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39fb2af24c9446068cb639312bd49dd9" candidate_uuid: "0be07e30136e46bba4bd70ed021b5fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06231b35aef94c21aa17f67a4ce44db6" is_pre_election: true
I20260927 18:39:00.259383 14454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39fb2af24c9446068cb639312bd49dd9" candidate_uuid: "0be07e30136e46bba4bd70ed021b5fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f4d38fc01f453b9f5369a816fc3c59" is_pre_election: true
I20260927 18:39:00.259470 14454 raft_consensus.cc:2454] T 39fb2af24c9446068cb639312bd49dd9 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0be07e30136e46bba4bd70ed021b5fbe in term 0.
I20260927 18:39:00.259496 14314 raft_consensus.cc:2454] T 39fb2af24c9446068cb639312bd49dd9 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0be07e30136e46bba4bd70ed021b5fbe in term 0.
I20260927 18:39:00.259768 14106 leader_election.cc:302] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0be07e30136e46bba4bd70ed021b5fbe, 96f4d38fc01f453b9f5369a816fc3c59; no voters:
I20260927 18:39:00.259917 14591 raft_consensus.cc:2781] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:39:00.260023 14591 raft_consensus.cc:482] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:39:00.260056 14591 raft_consensus.cc:3037] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:00.260848 14591 raft_consensus.cc:504] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:39:00.261045 14591 leader_election.cc:288] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe [CANDIDATE]: Term 1 election: Requested vote from peers 06231b35aef94c21aa17f67a4ce44db6 (127.11.251.194:41337), 96f4d38fc01f453b9f5369a816fc3c59 (127.11.251.195:38547)
I20260927 18:39:00.261225 14454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39fb2af24c9446068cb639312bd49dd9" candidate_uuid: "0be07e30136e46bba4bd70ed021b5fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f4d38fc01f453b9f5369a816fc3c59"
I20260927 18:39:00.261231 14314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39fb2af24c9446068cb639312bd49dd9" candidate_uuid: "0be07e30136e46bba4bd70ed021b5fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06231b35aef94c21aa17f67a4ce44db6"
I20260927 18:39:00.261305 14314 raft_consensus.cc:3037] T 39fb2af24c9446068cb639312bd49dd9 P 06231b35aef94c21aa17f67a4ce44db6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:00.261308 14454 raft_consensus.cc:3037] T 39fb2af24c9446068cb639312bd49dd9 P 96f4d38fc01f453b9f5369a816fc3c59 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:00.262087 14454 raft_consensus.cc:2454] T 39fb2af24c9446068cb639312bd49dd9 P 96f4d38fc01f453b9f5369a816fc3c59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0be07e30136e46bba4bd70ed021b5fbe in term 1.
I20260927 18:39:00.262290 14314 raft_consensus.cc:2454] T 39fb2af24c9446068cb639312bd49dd9 P 06231b35aef94c21aa17f67a4ce44db6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0be07e30136e46bba4bd70ed021b5fbe in term 1.
I20260927 18:39:00.262331 14106 leader_election.cc:302] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0be07e30136e46bba4bd70ed021b5fbe, 96f4d38fc01f453b9f5369a816fc3c59; no voters:
I20260927 18:39:00.262456 14591 raft_consensus.cc:2781] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:00.262549 14591 raft_consensus.cc:686] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe [term 1 LEADER]: Becoming Leader. State: Replica: 0be07e30136e46bba4bd70ed021b5fbe, State: Running, Role: LEADER
I20260927 18:39:00.262619 14591 consensus_queue.cc:237] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } }
I20260927 18:39:00.263386 14015 catalog_manager.cc:5495] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 0be07e30136e46bba4bd70ed021b5fbe (127.11.251.193). New cstate: current_term: 1 leader_uuid: "0be07e30136e46bba4bd70ed021b5fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be07e30136e46bba4bd70ed021b5fbe" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 } health_report { overall_health: UNKNOWN } } }
I20260927 18:39:00.302814 14634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:00.300358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51720 (local address 127.11.251.193:39889)
0927 18:39:00.300581 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:00.300582 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:00.300589 (+ 7us) server_negotiation.cc:407] Connection header received
0927 18:39:00.300606 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:00.300607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:00.300617 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:00.300670 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:39:00.300829 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:00.301312 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:00.302154 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:00.302343 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:00.302408 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:00.302532 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:39:00.302574 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:39:00.302616 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:00.302675 (+ 59us) server_negotiation.cc:299] Negotiation successful
0927 18:39:00.302691 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":150,"threads_started":1}
I20260927 18:39:00.307895 14634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:00.305783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:52739 (local address 127.11.251.193:39889)
0927 18:39:00.305832 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:00.305834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:00.305843 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:39:00.305888 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:00.305889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:00.305898 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:00.305950 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:39:00.306099 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:00.306546 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:00.307265 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:00.307454 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:00.307534 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:00.307654 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:39:00.307684 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:39:00.307739 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:00.307791 (+ 52us) server_negotiation.cc:299] Negotiation successful
0927 18:39:00.307811 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260927 18:39:00.307909 14637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:00.305615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:39889 (local address 127.11.251.193:52739)
0927 18:39:00.305757 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:00.305768 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:00.305830 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:00.305957 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:00.305958 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:00.305959 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:39:00.306045 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:00.306046 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:00.306621 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:00.306621 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:00.307181 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:00.307183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:00.307535 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:00.307536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:00.307586 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:00.307590 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:39:00.307740 (+ 150us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:39:00.307742 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:39:00.307789 (+ 47us) client_negotiation.cc:240] Negotiation successful
0927 18:39:00.307832 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":1,"thread_start_us":92,"threads_started":1}
I20260927 18:39:00.310200 14454 raft_consensus.cc:1261] T 39fb2af24c9446068cb639312bd49dd9 P 96f4d38fc01f453b9f5369a816fc3c59 [term 1 FOLLOWER]: Refusing update from remote peer 0be07e30136e46bba4bd70ed021b5fbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:00.310209 14314 raft_consensus.cc:1261] T 39fb2af24c9446068cb639312bd49dd9 P 06231b35aef94c21aa17f67a4ce44db6 [term 1 FOLLOWER]: Refusing update from remote peer 0be07e30136e46bba4bd70ed021b5fbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:00.310475 14557 consensus_queue.cc:1035] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "96f4d38fc01f453b9f5369a816fc3c59" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:00.310604 14591 consensus_queue.cc:1035] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "06231b35aef94c21aa17f67a4ce44db6" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 18:39:00.315488 14583 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260927 18:39:01.839679 14017 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:01.839819 14017 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:01.840783 14017 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f895b58d92d549a5a016d57ef7750545: Sending DeleteTablet for 3 replicas of tablet 39fb2af24c9446068cb639312bd49dd9
I20260927 18:39:01.841074 14154 tablet_service.cc:1514] Processing DeleteTablet for tablet 39fb2af24c9446068cb639312bd49dd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:01 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:51706
I20260927 18:39:01.841087 14421 tablet_service.cc:1514] Processing DeleteTablet for tablet 39fb2af24c9446068cb639312bd49dd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:01 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:47012
I20260927 18:39:01.841284 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 14089
I20260927 18:39:01.841343 14661 tablet_replica.cc:330] T 39fb2af24c9446068cb639312bd49dd9 P 0be07e30136e46bba4bd70ed021b5fbe: stopping tablet replica
I20260927 18:39:01.841873 14294 tablet_service.cc:1514] Processing DeleteTablet for tablet 39fb2af24c9446068cb639312bd49dd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:01 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:33680
I20260927 18:39:01.842072 14663 tablet_replica.cc:330] T 39fb2af24c9446068cb639312bd49dd9 P 06231b35aef94c21aa17f67a4ce44db6: stopping tablet replica
I20260927 18:39:01.842136 14663 raft_consensus.cc:2229] T 39fb2af24c9446068cb639312bd49dd9 P 06231b35aef94c21aa17f67a4ce44db6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:01.842219 14663 raft_consensus.cc:2258] T 39fb2af24c9446068cb639312bd49dd9 P 06231b35aef94c21aa17f67a4ce44db6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:01.842506 14663 ts_tablet_manager.cc:1905] T 39fb2af24c9446068cb639312bd49dd9 P 06231b35aef94c21aa17f67a4ce44db6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:01.844597 14663 ts_tablet_manager.cc:1918] T 39fb2af24c9446068cb639312bd49dd9 P 06231b35aef94c21aa17f67a4ce44db6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 18:39:01.844666 14663 log.cc:1196] T 39fb2af24c9446068cb639312bd49dd9 P 06231b35aef94c21aa17f67a4ce44db6: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/39fb2af24c9446068cb639312bd49dd9
I20260927 18:39:01.844894 14663 ts_tablet_manager.cc:1939] T 39fb2af24c9446068cb639312bd49dd9 P 06231b35aef94c21aa17f67a4ce44db6: Deleting consensus metadata
I20260927 18:39:01.845273 14004 catalog_manager.cc:4841] TS 06231b35aef94c21aa17f67a4ce44db6 (127.11.251.194:41337): tablet 39fb2af24c9446068cb639312bd49dd9 (table test-table [id=deb47fb249e048ddb7171dbeb4aa0f09]) successfully deleted
I20260927 18:39:01.845728 14662 tablet_replica.cc:330] T 39fb2af24c9446068cb639312bd49dd9 P 96f4d38fc01f453b9f5369a816fc3c59: stopping tablet replica
I20260927 18:39:01.845808 14662 raft_consensus.cc:2229] T 39fb2af24c9446068cb639312bd49dd9 P 96f4d38fc01f453b9f5369a816fc3c59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:01.845876 14662 raft_consensus.cc:2258] T 39fb2af24c9446068cb639312bd49dd9 P 96f4d38fc01f453b9f5369a816fc3c59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:01.846139 14662 ts_tablet_manager.cc:1905] T 39fb2af24c9446068cb639312bd49dd9 P 96f4d38fc01f453b9f5369a816fc3c59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:01.847651 14662 ts_tablet_manager.cc:1918] T 39fb2af24c9446068cb639312bd49dd9 P 96f4d38fc01f453b9f5369a816fc3c59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 18:39:01.847707 14662 log.cc:1196] T 39fb2af24c9446068cb639312bd49dd9 P 96f4d38fc01f453b9f5369a816fc3c59: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/39fb2af24c9446068cb639312bd49dd9
I20260927 18:39:01.847947 14662 ts_tablet_manager.cc:1939] T 39fb2af24c9446068cb639312bd49dd9 P 96f4d38fc01f453b9f5369a816fc3c59: Deleting consensus metadata
I20260927 18:39:01.848356 14002 catalog_manager.cc:4841] TS 96f4d38fc01f453b9f5369a816fc3c59 (127.11.251.195:38547): tablet 39fb2af24c9446068cb639312bd49dd9 (table test-table [id=deb47fb249e048ddb7171dbeb4aa0f09]) successfully deleted
W20260927 18:39:01.850347 14001 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.251.193:39889 (error 108)
I20260927 18:39:01.850590 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 14229
I20260927 18:39:01.851528 14665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:01.850771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:39889 (local address 127.0.0.1:51728)
0927 18:39:01.851392 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:01.851470 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.193:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":59,"threads_started":1}
W20260927 18:39:01.851711 14001 catalog_manager.cc:4584] TS 0be07e30136e46bba4bd70ed021b5fbe (127.11.251.193:39889): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 39fb2af24c9446068cb639312bd49dd9: Network error: Client connection negotiation failed: client connection to 127.11.251.193:39889: connect: Connection refused (error 111)
I20260927 18:39:01.856948 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 14369
I20260927 18:39:01.864481 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 13985
2026-09-27T18:39:01Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5553 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:39:01 dist-test-slave-z47j krb5kdc[14668](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:39:01 dist-test-slave-z47j krb5kdc[14668](info): set up 2 sockets
Sep 27 18:39:01 dist-test-slave-z47j krb5kdc[14668](info): commencing operation
krb5kdc: starting...
W20260927 18:39:02.922240 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:39:03 dist-test-slave-z47j krb5kdc[14668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:39:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:39:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:39:03.060462 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:40115
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:40115
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:03.142781 14681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:03.143014 14681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:03.143050 14681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:03.144220 14681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:03.144286 14681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:03.144312 14681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:03.144342 14681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:03.144372 14681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:39:03.145929 14681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:40115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:40115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:03.146186 14681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:03.146453 14681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:03.151526 14687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:03.151582 14686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:03.151527 14689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:03.151934 14681 server_base.cc:1034] running on GCE node
I20260927 18:39:03.152190 14681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:03.152544 14681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:03.153689 14681 hybrid_clock.cc:648] HybridClock initialized: now 1790534343153657 us; error 47 us; skew 500 ppm
Sep 27 18:39:03 dist-test-slave-z47j krb5kdc[14668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:03.156863 14681 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:39:03.157428 14681 webserver.cc:460] Webserver started at http://127.11.251.254:35899/ using document root <none> and password file <none>
I20260927 18:39:03.157738 14681 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:03.157796 14681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:03.157970 14681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:03.159348 14681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "4a07a649198344abb2c427a0d3b6b720"
format_stamp: "Formatted at 2026-09-27 18:39:03 on dist-test-slave-z47j"
server_key: "05cff6693f7a473ed9a5db91493c60ba"
server_key_iv: "b1a054d28ecddb91a7c55a58290b665d"
server_key_version: "encryptionkey@0"
I20260927 18:39:03.159801 14681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "4a07a649198344abb2c427a0d3b6b720"
format_stamp: "Formatted at 2026-09-27 18:39:03 on dist-test-slave-z47j"
server_key: "05cff6693f7a473ed9a5db91493c60ba"
server_key_iv: "b1a054d28ecddb91a7c55a58290b665d"
server_key_version: "encryptionkey@0"
I20260927 18:39:03.161522 14681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260927 18:39:03.162555 14696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:03.162849 14681 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:39:03.162997 14681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "4a07a649198344abb2c427a0d3b6b720"
format_stamp: "Formatted at 2026-09-27 18:39:03 on dist-test-slave-z47j"
server_key: "05cff6693f7a473ed9a5db91493c60ba"
server_key_iv: "b1a054d28ecddb91a7c55a58290b665d"
server_key_version: "encryptionkey@0"
I20260927 18:39:03.163086 14681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:03.179713 14681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:03.182591 14681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:03.182789 14681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:03.188141 14681 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:40115
I20260927 18:39:03.188236 14748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:40115 every 8 connection(s)
I20260927 18:39:03.188831 14681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:39:03.189986 14749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:03.192422 14749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720: Bootstrap starting.
I20260927 18:39:03.193060 14749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:03.193372 14749 log.cc:824] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:03.194310 14749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720: No bootstrap required, opened a new log
I20260927 18:39:03.195963 14749 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a07a649198344abb2c427a0d3b6b720" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40115 } }
I20260927 18:39:03.196115 14749 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:03.196136 14749 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a07a649198344abb2c427a0d3b6b720, State: Initialized, Role: FOLLOWER
I20260927 18:39:03.196300 14749 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a07a649198344abb2c427a0d3b6b720" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40115 } }
I20260927 18:39:03.196401 14749 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:39:03.196437 14749 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:39:03.196483 14749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:03.196511 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 14681
I20260927 18:39:03.196628 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:39:03.196866 12271 external_mini_cluster.cc:1428] Setting key 2fe5dc4315506d14f38ff1bb63164a90
I20260927 18:39:03.197232 14749 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a07a649198344abb2c427a0d3b6b720" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40115 } }
I20260927 18:39:03.197368 14749 leader_election.cc:302] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a07a649198344abb2c427a0d3b6b720; no voters:
I20260927 18:39:03.197660 14749 leader_election.cc:288] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:39:03.197937 14753 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:03.198112 14753 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720 [term 1 LEADER]: Becoming Leader. State: Replica: 4a07a649198344abb2c427a0d3b6b720, State: Running, Role: LEADER
I20260927 18:39:03.198129 14749 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:03.198985 14753 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a07a649198344abb2c427a0d3b6b720" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40115 } }
I20260927 18:39:03.199766 14756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a07a649198344abb2c427a0d3b6b720. Latest consensus state: current_term: 1 leader_uuid: "4a07a649198344abb2c427a0d3b6b720" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a07a649198344abb2c427a0d3b6b720" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40115 } } }
I20260927 18:39:03.199939 14755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a07a649198344abb2c427a0d3b6b720" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a07a649198344abb2c427a0d3b6b720" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40115 } } }
I20260927 18:39:03.200073 14756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:03.200079 14755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720 [sys.catalog]: This master's current role is: LEADER
W20260927 18:39:03.200559 14768 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:39:03.200626 14768 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:39:03.200838 14769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:39:03.201006 14769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:39:03.202831 14769 catalog_manager.cc:1347] Generated new cluster ID: fdd11d3e11ec49758a68b2d0aa1f111e
I20260927 18:39:03.202899 14769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:39:03 dist-test-slave-z47j krb5kdc[14668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:03.209621 14754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:03.197567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36526 (local address 127.11.251.254:40115)
0927 18:39:03.198173 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:03.198187 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:03.198249 (+ 62us) server_negotiation.cc:407] Connection header received
0927 18:39:03.198431 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:03.198440 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:03.198509 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:03.198723 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:03.199614 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:03.201123 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:03.202227 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:03.202876 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:03.202997 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:03.205653 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:03.205658 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:03.205659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:03.205672 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:03.207952 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:03.208349 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:03.208353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:03.208354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:03.208446 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:03.208652 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:03.208652 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:03.208653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:03.208938 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:03.209137 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:03.209217 (+ 80us) server_negotiation.cc:299] Negotiation successful
0927 18:39:03.209383 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":153,"threads_started":1}
I20260927 18:39:03.224581 14769 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:39:03.225238 14769 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:39:03.231086 14769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a07a649198344abb2c427a0d3b6b720: Generated new TSK 0
I20260927 18:39:03.231320 14769 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.251.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
I20260927 18:39:03.261445 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:40115
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:03.353992 14775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:03.354247 14775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:03.354272 14775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:03.355506 14775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:03.355542 14775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:03.355568 14775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:39:03.357137 14775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:40115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:03.357363 14775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:03.357605 14775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:39:03.363413 14775 server_base.cc:1034] running on GCE node
W20260927 18:39:03.363350 14780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:03.363353 14783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:03.363350 14781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:03.363897 14775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:03.364215 14775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:03.365417 14775 hybrid_clock.cc:648] HybridClock initialized: now 1790534343365398 us; error 41 us; skew 500 ppm
Sep 27 18:39:03 dist-test-slave-z47j krb5kdc[14668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:03.368629 14775 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:39:03.369199 14775 webserver.cc:460] Webserver started at http://127.11.251.193:42569/ using document root <none> and password file <none>
I20260927 18:39:03.369465 14775 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:03.369534 14775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:03.369717 14775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:03.370958 14775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "919fa62dc1174305bca8f8bd325c025f"
format_stamp: "Formatted at 2026-09-27 18:39:03 on dist-test-slave-z47j"
server_key: "9e45b2fe2d577af33f70ff9464aa12bd"
server_key_iv: "75d6ab3d359db32ea04d1acaa2729b2d"
server_key_version: "encryptionkey@0"
I20260927 18:39:03.371379 14775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "919fa62dc1174305bca8f8bd325c025f"
format_stamp: "Formatted at 2026-09-27 18:39:03 on dist-test-slave-z47j"
server_key: "9e45b2fe2d577af33f70ff9464aa12bd"
server_key_iv: "75d6ab3d359db32ea04d1acaa2729b2d"
server_key_version: "encryptionkey@0"
I20260927 18:39:03.372942 14775 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:39:03.373926 14790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:03.374116 14775 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:03.374233 14775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "919fa62dc1174305bca8f8bd325c025f"
format_stamp: "Formatted at 2026-09-27 18:39:03 on dist-test-slave-z47j"
server_key: "9e45b2fe2d577af33f70ff9464aa12bd"
server_key_iv: "75d6ab3d359db32ea04d1acaa2729b2d"
server_key_version: "encryptionkey@0"
I20260927 18:39:03.374341 14775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:03.401464 14775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:03.404064 14775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:03.404245 14775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:03.404538 14775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:03.404901 14775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:03.404953 14775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:03.405014 14775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:03.405045 14775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:03.412853 14775 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:40777
I20260927 18:39:03.413421 14775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:39:03.413610 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 14775
I20260927 18:39:03.413719 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:39:03.413985 12271 external_mini_cluster.cc:1428] Setting key b46f98d4077d50d9155ad5be4e803897
I20260927 18:39:03.415060 14903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:40777 every 8 connection(s)
Sep 27 18:39:03 dist-test-slave-z47j krb5kdc[14668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.251.194@KRBTEST.COM; defaulting to no policy
I20260927 18:39:03.426275 14754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:03.414459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:44403 (local address 127.11.251.254:40115)
0927 18:39:03.414530 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:03.414534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:03.415633 (+ 1099us) server_negotiation.cc:407] Connection header received
0927 18:39:03.416516 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:03.416517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:03.416533 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:03.416565 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:03.418072 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:03.418631 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:03.419661 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:03.419988 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:03.420067 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:03.422742 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:03.422744 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:03.422745 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:03.422760 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:03.424670 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:03.425131 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:03.425132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:03.425133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:03.425202 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:03.425463 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:03.425464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:03.425465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:03.425660 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:03.425737 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:03.426032 (+ 295us) server_negotiation.cc:299] Negotiation successful
0927 18:39:03.426143 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 18:39:03.426421 14906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:03.414561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:40115 (local address 127.11.251.193:44403)
0927 18:39:03.415466 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:03.415523 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:03.416389 (+ 866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:03.416759 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:03.416764 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:03.417615 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:03.417947 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:03.417953 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:03.418724 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:03.418724 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:03.419569 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:03.419571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:03.420113 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:03.420115 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:03.420244 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:03.420923 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:03.420932 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:03.422645 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:03.424770 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:03.424777 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:03.424779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:03.425051 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:03.425277 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:03.425278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:03.425278 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:03.425392 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:03.425726 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:03.425732 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:03.425891 (+ 159us) client_negotiation.cc:769] Sending connection context
0927 18:39:03.425988 (+ 97us) client_negotiation.cc:240] Negotiation successful
0927 18:39:03.426161 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"thread_start_us":97,"threads_started":1}
I20260927 18:39:03.426998 14904 heartbeater.cc:347] Connected to a master server at 127.11.251.254:40115
I20260927 18:39:03.427172 14904 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:03.427407 14904 heartbeater.cc:511] Master 127.11.251.254:40115 requested a full tablet report, sending...
I20260927 18:39:03.427971 14713 ts_manager.cc:194] Registered new tserver with Master: 919fa62dc1174305bca8f8bd325c025f (127.11.251.193:40777)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.194@KRBTEST.COM" created.
I20260927 18:39:03.428959 14713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:44403
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
I20260927 18:39:03.448163 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:40115
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:03.540585 14909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:03.540825 14909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:03.540855 14909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:03.542153 14909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:03.542193 14909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:03.542220 14909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:39:03.543805 14909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:40115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:03.544056 14909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:03.544287 14909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:03.550417 14915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:03.550457 14917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:03.550427 14914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:03.550647 14909 server_base.cc:1034] running on GCE node
I20260927 18:39:03.551079 14909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:03.551446 14909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:03.552649 14909 hybrid_clock.cc:648] HybridClock initialized: now 1790534343552632 us; error 43 us; skew 500 ppm
Sep 27 18:39:03 dist-test-slave-z47j krb5kdc[14668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:03.555840 14909 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:39:03.556444 14909 webserver.cc:460] Webserver started at http://127.11.251.194:42707/ using document root <none> and password file <none>
I20260927 18:39:03.556722 14909 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:03.556790 14909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:03.556953 14909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:03.558281 14909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "a0a1cc824b024f49a84ab2a3b49fd192"
format_stamp: "Formatted at 2026-09-27 18:39:03 on dist-test-slave-z47j"
server_key: "990f9ee8a854e0e8091066ec7e43815a"
server_key_iv: "8785f46c60d7ca58d2aa8fb6344d7d41"
server_key_version: "encryptionkey@0"
I20260927 18:39:03.558710 14909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "a0a1cc824b024f49a84ab2a3b49fd192"
format_stamp: "Formatted at 2026-09-27 18:39:03 on dist-test-slave-z47j"
server_key: "990f9ee8a854e0e8091066ec7e43815a"
server_key_iv: "8785f46c60d7ca58d2aa8fb6344d7d41"
server_key_version: "encryptionkey@0"
I20260927 18:39:03.560179 14909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:39:03.561066 14924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:03.561264 14909 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:03.561403 14909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "a0a1cc824b024f49a84ab2a3b49fd192"
format_stamp: "Formatted at 2026-09-27 18:39:03 on dist-test-slave-z47j"
server_key: "990f9ee8a854e0e8091066ec7e43815a"
server_key_iv: "8785f46c60d7ca58d2aa8fb6344d7d41"
server_key_version: "encryptionkey@0"
I20260927 18:39:03.561482 14909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:03.576459 14909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:03.579694 14909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:03.579890 14909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:03.580423 14909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:03.581244 14909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:03.581331 14909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:03.581450 14909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:03.581501 14909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:03.592033 14909 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:38955
I20260927 18:39:03.592080 15037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:38955 every 8 connection(s)
I20260927 18:39:03.592504 14909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:39:03.594214 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 14909
I20260927 18:39:03.594326 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:39:03.594561 12271 external_mini_cluster.cc:1428] Setting key b325b4c2827ecac2233a4cc65469ab70
Sep 27 18:39:03 dist-test-slave-z47j krb5kdc[14668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.251.195@KRBTEST.COM; defaulting to no policy
I20260927 18:39:03.605769 14754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:03.593437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:51561 (local address 127.11.251.254:40115)
0927 18:39:03.593518 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:03.593520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:03.593791 (+ 271us) server_negotiation.cc:407] Connection header received
0927 18:39:03.594660 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:03.594661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:03.594676 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:03.594742 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:03.596291 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:03.597034 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:03.598479 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:03.598787 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:03.598878 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:03.602027 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:03.602029 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:03.602029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:03.602037 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:03.603738 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:03.604396 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:03.604398 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:03.604398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:03.604469 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:03.604830 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:03.604831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:03.604832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:03.605019 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:03.605079 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:03.605460 (+ 381us) server_negotiation.cc:299] Negotiation successful
0927 18:39:03.605554 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:39:03.605927 15040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:03.593463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:40115 (local address 127.11.251.194:51561)
0927 18:39:03.593664 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:03.593707 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:03.594498 (+ 791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:03.594848 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:03.594852 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:03.595815 (+ 963us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:03.596166 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:03.596174 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:03.597175 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:03.597177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:03.598370 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:03.598373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:03.598890 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:03.598891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:03.599057 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:03.599868 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:03.599881 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:03.601906 (+ 2025us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:03.603849 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:03.603859 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:03.603863 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:03.604261 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:03.604545 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:03.604545 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:03.604546 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:03.604716 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:03.605110 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:03.605116 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:03.605274 (+ 158us) client_negotiation.cc:769] Sending connection context
0927 18:39:03.605367 (+ 93us) client_negotiation.cc:240] Negotiation successful
0927 18:39:03.605530 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":81,"threads_started":1}
I20260927 18:39:03.606528 15038 heartbeater.cc:347] Connected to a master server at 127.11.251.254:40115
I20260927 18:39:03.606633 15038 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:03.606830 15038 heartbeater.cc:511] Master 127.11.251.254:40115 requested a full tablet report, sending...
I20260927 18:39:03.607242 14713 ts_manager.cc:194] Registered new tserver with Master: a0a1cc824b024f49a84ab2a3b49fd192 (127.11.251.194:38955)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.195@KRBTEST.COM" created.
I20260927 18:39:03.607827 14713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:51561
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
I20260927 18:39:03.629501 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:40115
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:03.726540 15043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:03.726810 15043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:03.726848 15043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:03.728199 15043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:03.728243 15043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:03.728276 15043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:39:03.729993 15043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:40115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:03.730239 15043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:03.730470 15043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:03.736160 15051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:03.736126 15049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:03.736264 15043 server_base.cc:1034] running on GCE node
W20260927 18:39:03.736161 15048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:03.736581 15043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:03.736845 15043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:03.737990 15043 hybrid_clock.cc:648] HybridClock initialized: now 1790534343737972 us; error 42 us; skew 500 ppm
Sep 27 18:39:03 dist-test-slave-z47j krb5kdc[14668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:03.741269 15043 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:39:03.741892 15043 webserver.cc:460] Webserver started at http://127.11.251.195:45251/ using document root <none> and password file <none>
I20260927 18:39:03.742177 15043 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:03.742255 15043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:03.742409 15043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:03.743680 15043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "e51c1f04b3014f32afbe7ef736b409c6"
format_stamp: "Formatted at 2026-09-27 18:39:03 on dist-test-slave-z47j"
server_key: "3239540bf5d35b9d12985a2f2293191b"
server_key_iv: "87158ecb63e66357186aa5fd4c4031bb"
server_key_version: "encryptionkey@0"
I20260927 18:39:03.744092 15043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "e51c1f04b3014f32afbe7ef736b409c6"
format_stamp: "Formatted at 2026-09-27 18:39:03 on dist-test-slave-z47j"
server_key: "3239540bf5d35b9d12985a2f2293191b"
server_key_iv: "87158ecb63e66357186aa5fd4c4031bb"
server_key_version: "encryptionkey@0"
I20260927 18:39:03.745750 15043 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260927 18:39:03.746598 15058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:03.746788 15043 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:03.746886 15043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "e51c1f04b3014f32afbe7ef736b409c6"
format_stamp: "Formatted at 2026-09-27 18:39:03 on dist-test-slave-z47j"
server_key: "3239540bf5d35b9d12985a2f2293191b"
server_key_iv: "87158ecb63e66357186aa5fd4c4031bb"
server_key_version: "encryptionkey@0"
I20260927 18:39:03.746963 15043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:03.756673 15043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:03.759155 15043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:03.759326 15043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:03.759629 15043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:03.759968 15043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:03.760018 15043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:03.760061 15043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:03.760088 15043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:03.766924 15043 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:33503
I20260927 18:39:03.766981 15171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:33503 every 8 connection(s)
I20260927 18:39:03.767321 15043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:39:03.776366 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 15043
I20260927 18:39:03.776525 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:39:03.776757 12271 external_mini_cluster.cc:1428] Setting key 18137e21dff971b738b2700508b93331
Sep 27 18:39:03 dist-test-slave-z47j krb5kdc[14668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:03.784116 14754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:03.768109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:32931 (local address 127.11.251.254:40115)
0927 18:39:03.768188 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:03.768190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:03.768430 (+ 240us) server_negotiation.cc:407] Connection header received
0927 18:39:03.769064 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:03.769065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:03.769077 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:03.769137 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:03.770371 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:03.770990 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:03.772698 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:03.772934 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:03.773365 (+ 431us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:03.778182 (+ 4817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:03.778185 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:03.778185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:03.778196 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:03.779901 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:03.781450 (+ 1549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:03.781452 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:03.781452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:03.781530 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:03.782707 (+ 1177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:03.782709 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:03.782709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:03.782918 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:03.783001 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:03.783919 (+ 918us) server_negotiation.cc:299] Negotiation successful
0927 18:39:03.784026 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 18:39:03.784677 15174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:03.768101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:40115 (local address 127.11.251.195:32931)
0927 18:39:03.768291 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:03.768331 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:03.768884 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:03.769213 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:03.769217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:03.770001 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:03.770245 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:03.770251 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:03.771101 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:03.771102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:03.772305 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:03.772314 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:03.773525 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:03.773526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:03.773771 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:03.774701 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:03.774713 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:03.778053 (+ 3340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:03.780531 (+ 2478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:03.780541 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:03.780544 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:03.780909 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:03.782429 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:03.782430 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:03.782430 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:03.782589 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:03.783107 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:03.783117 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:03.783548 (+ 431us) client_negotiation.cc:769] Sending connection context
0927 18:39:03.783848 (+ 300us) client_negotiation.cc:240] Negotiation successful
0927 18:39:03.784061 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":104,"threads_started":1}
I20260927 18:39:03.786903 15172 heartbeater.cc:347] Connected to a master server at 127.11.251.254:40115
I20260927 18:39:03.787045 15172 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:03.787284 15172 heartbeater.cc:511] Master 127.11.251.254:40115 requested a full tablet report, sending...
I20260927 18:39:03.787993 14713 ts_manager.cc:194] Registered new tserver with Master: e51c1f04b3014f32afbe7ef736b409c6 (127.11.251.195:33503)
I20260927 18:39:03.788457 14713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:32931
I20260927 18:39:03.790023 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 18:39:03.803310 14754 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:03.802218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36542 (local address 127.11.251.254:40115)
0927 18:39:03.802294 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:03.802296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:03.802343 (+ 47us) server_negotiation.cc:407] Connection header received
0927 18:39:03.802400 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:03.802401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:03.802413 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:03.802479 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:03.803199 (+ 720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36542: BlockingRecv error: recv got EOF from 127.0.0.1:36542 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:39:03.804241 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 14775
I20260927 18:39:03.809806 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 14909
I20260927 18:39:03.814877 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 15043
I20260927 18:39:03.820036 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 14681
2026-09-27T18:39:03Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1954 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:39:03 dist-test-slave-z47j krb5kdc[15184](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:39:03 dist-test-slave-z47j krb5kdc[15184](info): set up 2 sockets
Sep 27 18:39:03 dist-test-slave-z47j krb5kdc[15184](info): commencing operation
krb5kdc: starting...
W20260927 18:39:04.866598 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:39:04 dist-test-slave-z47j krb5kdc[15184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:39:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:39:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:39:04.997083 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:40329
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:40329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:05.098580 15197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:05.098883 15197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:05.098927 15197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:05.100157 15197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:05.100219 15197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:05.100247 15197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:05.100279 15197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:05.100303 15197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:39:05.102213 15197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:40329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:40329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:05.102560 15197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:05.102850 15197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:05.109974 15205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:05.109974 15202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:05.109982 15203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:05.110464 15197 server_base.cc:1034] running on GCE node
I20260927 18:39:05.110780 15197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:05.111132 15197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:05.112304 15197 hybrid_clock.cc:648] HybridClock initialized: now 1790534345112301 us; error 51 us; skew 500 ppm
Sep 27 18:39:05 dist-test-slave-z47j krb5kdc[15184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534345, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:05.115303 15197 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:39:05.115892 15197 webserver.cc:460] Webserver started at http://127.11.251.254:34045/ using document root <none> and password file <none>
I20260927 18:39:05.116158 15197 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:05.116228 15197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:05.116390 15197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:05.117692 15197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "0b0fd356efe444d1970fabd4f6cf1652"
format_stamp: "Formatted at 2026-09-27 18:39:05 on dist-test-slave-z47j"
server_key: "c7678a15d07f1af6f02ca528428ae7b3"
server_key_iv: "8aee38695429fcc3f70bbce20a8814f3"
server_key_version: "encryptionkey@0"
I20260927 18:39:05.118089 15197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "0b0fd356efe444d1970fabd4f6cf1652"
format_stamp: "Formatted at 2026-09-27 18:39:05 on dist-test-slave-z47j"
server_key: "c7678a15d07f1af6f02ca528428ae7b3"
server_key_iv: "8aee38695429fcc3f70bbce20a8814f3"
server_key_version: "encryptionkey@0"
I20260927 18:39:05.119591 15197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 18:39:05.120435 15212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:05.120621 15197 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:05.120734 15197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "0b0fd356efe444d1970fabd4f6cf1652"
format_stamp: "Formatted at 2026-09-27 18:39:05 on dist-test-slave-z47j"
server_key: "c7678a15d07f1af6f02ca528428ae7b3"
server_key_iv: "8aee38695429fcc3f70bbce20a8814f3"
server_key_version: "encryptionkey@0"
I20260927 18:39:05.120815 15197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:05.146262 15197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:05.148547 15197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:05.148728 15197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:05.153697 15197 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:40329
I20260927 18:39:05.153761 15264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:40329 every 8 connection(s)
I20260927 18:39:05.154330 15197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:39:05.155128 15265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:05.157805 15265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652: Bootstrap starting.
I20260927 18:39:05.158512 15265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:05.158854 15265 log.cc:824] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:05.159608 15265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652: No bootstrap required, opened a new log
I20260927 18:39:05.161360 15265 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0fd356efe444d1970fabd4f6cf1652" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40329 } }
I20260927 18:39:05.161525 15265 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:05.161548 15265 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b0fd356efe444d1970fabd4f6cf1652, State: Initialized, Role: FOLLOWER
I20260927 18:39:05.161746 15265 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0fd356efe444d1970fabd4f6cf1652" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40329 } }
I20260927 18:39:05.161825 15265 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:39:05.161849 15265 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:39:05.161918 15265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:05.162901 15265 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0fd356efe444d1970fabd4f6cf1652" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40329 } }
I20260927 18:39:05.163034 15265 leader_election.cc:302] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b0fd356efe444d1970fabd4f6cf1652; no voters:
I20260927 18:39:05.163038 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 15197
I20260927 18:39:05.163164 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:39:05.163256 15265 leader_election.cc:288] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:39:05.163300 15268 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:05.163378 12271 external_mini_cluster.cc:1428] Setting key ed4da03ffa5530dcda068f0268a0cd99
I20260927 18:39:05.163441 15268 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652 [term 1 LEADER]: Becoming Leader. State: Replica: 0b0fd356efe444d1970fabd4f6cf1652, State: Running, Role: LEADER
I20260927 18:39:05.163529 15268 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0fd356efe444d1970fabd4f6cf1652" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40329 } }
I20260927 18:39:05.163602 15265 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:05.164002 15268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b0fd356efe444d1970fabd4f6cf1652. Latest consensus state: current_term: 1 leader_uuid: "0b0fd356efe444d1970fabd4f6cf1652" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0fd356efe444d1970fabd4f6cf1652" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40329 } } }
I20260927 18:39:05.164089 15268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:05.164244 15269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b0fd356efe444d1970fabd4f6cf1652" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b0fd356efe444d1970fabd4f6cf1652" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40329 } } }
I20260927 18:39:05.164387 15269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:05.165032 15275 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:39:05.165727 15275 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:39:05.167290 15275 catalog_manager.cc:1347] Generated new cluster ID: 793c971ace954d27806acfb532307d0b
I20260927 18:39:05.167344 15275 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:39:05 dist-test-slave-z47j krb5kdc[15184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:05.175515 15275 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:39:05.176041 15273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:05.163957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39308 (local address 127.11.251.254:40329)
0927 18:39:05.164385 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:05.164395 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:05.164433 (+ 38us) server_negotiation.cc:407] Connection header received
0927 18:39:05.164571 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:05.164581 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:05.164638 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:05.164790 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:05.165722 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:05.167136 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:05.168277 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:05.168942 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:05.169056 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:05.172142 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:05.172146 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:05.172147 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:05.172160 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:05.174260 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:05.174678 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:05.174681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:05.174681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:05.174788 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:05.175022 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:05.175023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:05.175023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:05.175317 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:05.175490 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:05.175647 (+ 157us) server_negotiation.cc:299] Negotiation successful
0927 18:39:05.175795 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":83,"threads_started":1}
I20260927 18:39:05.176414 15275 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:39:05.177088 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:39:05.180251 15275 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b0fd356efe444d1970fabd4f6cf1652: Generated new TSK 0
I20260927 18:39:05.180441 15275 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:39:05.193253 15273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:05.189554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39320 (local address 127.11.251.254:40329)
0927 18:39:05.189649 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:05.189651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:05.189775 (+ 124us) server_negotiation.cc:407] Connection header received
0927 18:39:05.189869 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:05.189869 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:05.189888 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:05.189963 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:05.190394 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:05.190913 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:05.191696 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:05.191916 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:05.191993 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:05.192723 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:05.192724 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:05.192724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:05.192955 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:05.192971 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:05.192977 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:05.193046 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:05.193126 (+ 80us) server_negotiation.cc:299] Negotiation successful
0927 18:39:05.193154 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 18:39:05.195948 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 15197
2026-09-27T18:39:05Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1373 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:39:05 dist-test-slave-z47j krb5kdc[15298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:39:05 dist-test-slave-z47j krb5kdc[15298](info): set up 2 sockets
Sep 27 18:39:05 dist-test-slave-z47j krb5kdc[15298](info): commencing operation
krb5kdc: starting...
W20260927 18:39:06.241935 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:39:06 dist-test-slave-z47j krb5kdc[15298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:39:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:39:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:39:06.372586 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:32951
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:37883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:32951
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:06.458293 15311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:06.458521 15311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:06.458545 15311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:06.459766 15311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:06.459818 15311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:06.459834 15311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:06.459846 15311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:06.459858 15311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:39:06.461359 15311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:37883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:32951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:32951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:06.461650 15311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:06.461920 15311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:06.467000 15319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:06.467015 15316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:06.467167 15317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:06.467242 15311 server_base.cc:1034] running on GCE node
I20260927 18:39:06.467633 15311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:06.467957 15311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:06.469144 15311 hybrid_clock.cc:648] HybridClock initialized: now 1790534346469136 us; error 51 us; skew 500 ppm
Sep 27 18:39:06 dist-test-slave-z47j krb5kdc[15298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534346, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:06.472009 15311 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:39:06.472524 15311 webserver.cc:460] Webserver started at http://127.11.251.254:43011/ using document root <none> and password file <none>
I20260927 18:39:06.472765 15311 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:06.472848 15311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:06.473002 15311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:06.474164 15311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "515b28c485b34f0c966fe8d368cda1c1"
format_stamp: "Formatted at 2026-09-27 18:39:06 on dist-test-slave-z47j"
server_key: "3e16cb9b901be53a9defeb58580adf3e"
server_key_iv: "3dceb43c02bb203bef14ab13f9425b31"
server_key_version: "encryptionkey@0"
I20260927 18:39:06.474538 15311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "515b28c485b34f0c966fe8d368cda1c1"
format_stamp: "Formatted at 2026-09-27 18:39:06 on dist-test-slave-z47j"
server_key: "3e16cb9b901be53a9defeb58580adf3e"
server_key_iv: "3dceb43c02bb203bef14ab13f9425b31"
server_key_version: "encryptionkey@0"
I20260927 18:39:06.475967 15311 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:39:06.476801 15326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:06.477124 15311 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:06.477258 15311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "515b28c485b34f0c966fe8d368cda1c1"
format_stamp: "Formatted at 2026-09-27 18:39:06 on dist-test-slave-z47j"
server_key: "3e16cb9b901be53a9defeb58580adf3e"
server_key_iv: "3dceb43c02bb203bef14ab13f9425b31"
server_key_version: "encryptionkey@0"
I20260927 18:39:06.477339 15311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:06.489184 15311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:06.491487 15311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:06.491690 15311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:06.496248 15311 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:32951
I20260927 18:39:06.496313 15378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:32951 every 8 connection(s)
I20260927 18:39:06.496695 15311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:39:06.497910 15379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:06.498070 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 15311
I20260927 18:39:06.498231 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:39:06.498430 12271 external_mini_cluster.cc:1428] Setting key 143ce1b1ba31cf10b7c5c1727220f514
I20260927 18:39:06.502115 15379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1: Bootstrap starting.
I20260927 18:39:06.502916 15379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:06.503245 15379 log.cc:824] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:06.504163 15379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1: No bootstrap required, opened a new log
Sep 27 18:39:06 dist-test-slave-z47j krb5kdc[15298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:06.506534 15379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "515b28c485b34f0c966fe8d368cda1c1" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 32951 } }
I20260927 18:39:06.506747 15379 raft_consensus.cc:374] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:06.506786 15379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 515b28c485b34f0c966fe8d368cda1c1, State: Initialized, Role: FOLLOWER
I20260927 18:39:06.506963 15379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "515b28c485b34f0c966fe8d368cda1c1" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 32951 } }
I20260927 18:39:06.507074 15379 raft_consensus.cc:388] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:39:06.507138 15379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:39:06.507198 15379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:06.508122 15379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "515b28c485b34f0c966fe8d368cda1c1" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 32951 } }
I20260927 18:39:06.508247 15379 leader_election.cc:302] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 515b28c485b34f0c966fe8d368cda1c1; no voters:
I20260927 18:39:06.508436 15379 leader_election.cc:288] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:39:06.508529 15384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:06.508747 15379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:06.508757 15384 raft_consensus.cc:686] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1 [term 1 LEADER]: Becoming Leader. State: Replica: 515b28c485b34f0c966fe8d368cda1c1, State: Running, Role: LEADER
I20260927 18:39:06.509042 15384 consensus_queue.cc:237] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "515b28c485b34f0c966fe8d368cda1c1" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 32951 } }
I20260927 18:39:06.509580 15386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 515b28c485b34f0c966fe8d368cda1c1. Latest consensus state: current_term: 1 leader_uuid: "515b28c485b34f0c966fe8d368cda1c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "515b28c485b34f0c966fe8d368cda1c1" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 32951 } } }
I20260927 18:39:06.509766 15386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:06.509996 15385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "515b28c485b34f0c966fe8d368cda1c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "515b28c485b34f0c966fe8d368cda1c1" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 32951 } } }
I20260927 18:39:06.510142 15385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:06.510342 15393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:39:06.510947 15393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:39:06.511368 15382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:06.499175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34492 (local address 127.11.251.254:32951)
0927 18:39:06.499421 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:06.499433 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:06.499476 (+ 43us) server_negotiation.cc:407] Connection header received
0927 18:39:06.500039 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:06.500049 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:06.500122 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:06.500308 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:06.500934 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:06.502486 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:06.503385 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:06.503975 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:06.504102 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:06.506548 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:06.506552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:06.506553 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:06.506568 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:06.508862 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:06.509367 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:06.509370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:06.509371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:06.509513 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:06.509824 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:06.509825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:06.509825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:06.510216 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:06.510996 (+ 780us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:06.511079 (+ 83us) server_negotiation.cc:299] Negotiation successful
0927 18:39:06.511203 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":97,"threads_started":1}
I20260927 18:39:06.512669 15393 catalog_manager.cc:1347] Generated new cluster ID: 0b540dc3f3784ba582c811d3d54bcd04
I20260927 18:39:06.512733 15393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:39:06.542690 15393 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:39:06.543394 15393 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:39:06.554384 15393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 515b28c485b34f0c966fe8d368cda1c1: Generated new TSK 0
I20260927 18:39:06.554672 15393 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.251.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
I20260927 18:39:06.588969 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:32951
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:37883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:06.690788 15405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:06.691036 15405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:06.691061 15405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:06.692299 15405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:06.692348 15405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:06.692375 15405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:39:06.693990 15405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:37883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:32951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:06.694260 15405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:06.694581 15405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:06.700392 15410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:06.700513 15413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:06.700513 15411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:06.700562 15405 server_base.cc:1034] running on GCE node
I20260927 18:39:06.700963 15405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:06.701311 15405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:06.702471 15405 hybrid_clock.cc:648] HybridClock initialized: now 1790534346702451 us; error 40 us; skew 500 ppm
Sep 27 18:39:06 dist-test-slave-z47j krb5kdc[15298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534346, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:06.705264 15405 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:39:06.705834 15405 webserver.cc:460] Webserver started at http://127.11.251.193:35307/ using document root <none> and password file <none>
I20260927 18:39:06.706073 15405 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:06.706142 15405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:06.706300 15405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:06.707422 15405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "319f0895daae4651b608af5d50530e64"
format_stamp: "Formatted at 2026-09-27 18:39:06 on dist-test-slave-z47j"
server_key: "9e09ba7dae3e6105ae4a882bde52da80"
server_key_iv: "0e57850f5beeabc2b7f586b8d2dbf9e7"
server_key_version: "encryptionkey@0"
I20260927 18:39:06.707795 15405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "319f0895daae4651b608af5d50530e64"
format_stamp: "Formatted at 2026-09-27 18:39:06 on dist-test-slave-z47j"
server_key: "9e09ba7dae3e6105ae4a882bde52da80"
server_key_iv: "0e57850f5beeabc2b7f586b8d2dbf9e7"
server_key_version: "encryptionkey@0"
I20260927 18:39:06.709488 15405 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:39:06.710582 15420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:06.710856 15405 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:06.710971 15405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "319f0895daae4651b608af5d50530e64"
format_stamp: "Formatted at 2026-09-27 18:39:06 on dist-test-slave-z47j"
server_key: "9e09ba7dae3e6105ae4a882bde52da80"
server_key_iv: "0e57850f5beeabc2b7f586b8d2dbf9e7"
server_key_version: "encryptionkey@0"
I20260927 18:39:06.711066 15405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:06.745200 15405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:06.747800 15405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:06.747967 15405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:06.748277 15405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:06.748678 15405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:06.748733 15405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:06.748780 15405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:06.748811 15405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:06.756006 15405 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:35733
I20260927 18:39:06.756170 15533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:35733 every 8 connection(s)
I20260927 18:39:06.756500 15405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
Sep 27 18:39:06 dist-test-slave-z47j krb5kdc[15298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534346, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:06.766578 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 15405
I20260927 18:39:06.766716 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:39:06.766963 12271 external_mini_cluster.cc:1428] Setting key b423905784144b2f8460a201f478f0aa
I20260927 18:39:06.767027 15382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:06.757376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:52325 (local address 127.11.251.254:32951)
0927 18:39:06.757526 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:06.757529 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:06.757765 (+ 236us) server_negotiation.cc:407] Connection header received
0927 18:39:06.758362 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:06.758363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:06.758376 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:06.758434 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:06.759472 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:06.760021 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:06.761050 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:06.761269 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:06.761353 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:06.763648 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:06.763650 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:06.763651 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:06.763660 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:06.765143 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:06.765711 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:06.765712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:06.765712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:06.765774 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:06.766106 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:06.766107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:06.766107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:06.766249 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:06.766396 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:06.766829 (+ 433us) server_negotiation.cc:299] Negotiation successful
0927 18:39:06.766930 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260927 18:39:06.767139 15536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:06.757288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:32951 (local address 127.11.251.193:52325)
0927 18:39:06.757607 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:06.757658 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:06.758246 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:06.758522 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:06.758526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:06.759156 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:06.759364 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:06.759368 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:06.760118 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:06.760119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:06.760941 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:06.760943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:06.761366 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:06.761367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:06.761498 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:06.762041 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:06.762050 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:06.763525 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:06.765257 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:06.765263 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:06.765265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:06.765611 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:06.765854 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:06.765855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:06.765856 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:06.765986 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:06.766403 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:06.766409 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:06.766578 (+ 169us) client_negotiation.cc:769] Sending connection context
0927 18:39:06.766715 (+ 137us) client_negotiation.cc:240] Negotiation successful
0927 18:39:06.766890 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":110,"threads_started":1}
I20260927 18:39:06.767787 15534 heartbeater.cc:347] Connected to a master server at 127.11.251.254:32951
I20260927 18:39:06.767911 15534 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:06.768113 15534 heartbeater.cc:511] Master 127.11.251.254:32951 requested a full tablet report, sending...
I20260927 18:39:06.768769 15342 ts_manager.cc:194] Registered new tserver with Master: 319f0895daae4651b608af5d50530e64 (127.11.251.193:35733)
I20260927 18:39:06.769775 15342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:52325
WARNING: no policy specified for kudu/127.11.251.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
I20260927 18:39:06.794916 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:32951
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:37883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:06.877806 15539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:06.878055 15539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:06.878093 15539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:06.879245 15539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:06.879317 15539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:06.879362 15539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:39:06.880975 15539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:37883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:32951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:06.881249 15539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:06.881480 15539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:39:06.886852 15539 server_base.cc:1034] running on GCE node
W20260927 18:39:06.886852 15544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:06.886919 15547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:06.886987 15545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:06.887389 15539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:06.887633 15539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:06.888787 15539 hybrid_clock.cc:648] HybridClock initialized: now 1790534346888766 us; error 43 us; skew 500 ppm
Sep 27 18:39:06 dist-test-slave-z47j krb5kdc[15298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534346, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:06.891693 15539 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:39:06.892220 15539 webserver.cc:460] Webserver started at http://127.11.251.194:36915/ using document root <none> and password file <none>
I20260927 18:39:06.892463 15539 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:06.892534 15539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:06.892690 15539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:06.893817 15539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "48d46dec14cf4ad3b1cdf398d1db4c78"
format_stamp: "Formatted at 2026-09-27 18:39:06 on dist-test-slave-z47j"
server_key: "2b595a27502d0fbc62d7d9368220a4ed"
server_key_iv: "e9938172a6c32c3260c397bb485221ab"
server_key_version: "encryptionkey@0"
I20260927 18:39:06.894176 15539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "48d46dec14cf4ad3b1cdf398d1db4c78"
format_stamp: "Formatted at 2026-09-27 18:39:06 on dist-test-slave-z47j"
server_key: "2b595a27502d0fbc62d7d9368220a4ed"
server_key_iv: "e9938172a6c32c3260c397bb485221ab"
server_key_version: "encryptionkey@0"
I20260927 18:39:06.895526 15539 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:39:06.896338 15554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:06.896623 15539 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:06.896759 15539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "48d46dec14cf4ad3b1cdf398d1db4c78"
format_stamp: "Formatted at 2026-09-27 18:39:06 on dist-test-slave-z47j"
server_key: "2b595a27502d0fbc62d7d9368220a4ed"
server_key_iv: "e9938172a6c32c3260c397bb485221ab"
server_key_version: "encryptionkey@0"
I20260927 18:39:06.896849 15539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:06.947352 15539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:06.949514 15539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:06.949649 15539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:06.949935 15539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:06.950289 15539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:06.950330 15539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:06.950353 15539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:06.950407 15539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:06.956965 15539 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:39807
I20260927 18:39:06.957029 15667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:39807 every 8 connection(s)
I20260927 18:39:06.957338 15539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:39:06.961001 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 15539
I20260927 18:39:06.961143 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:39:06.961360 12271 external_mini_cluster.cc:1428] Setting key 0173700d7a07259648fdf31ca80a8ec7
Sep 27 18:39:06 dist-test-slave-z47j krb5kdc[15298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534346, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:06.968318 15670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:06.958068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:32951 (local address 127.11.251.194:49193)
0927 18:39:06.958272 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:06.958346 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:06.959000 (+ 654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:06.959275 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:06.959279 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:06.959945 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:06.960180 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:06.960185 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:06.960904 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:06.960906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:06.961987 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:06.961990 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:06.962419 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:06.962420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:06.962558 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:06.963149 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:06.963157 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:06.964783 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:06.966585 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:06.966595 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:06.966598 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:06.966982 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:06.967229 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:06.967230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:06.967230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:06.967389 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:06.967772 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:06.967776 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:06.967903 (+ 127us) client_negotiation.cc:769] Sending connection context
0927 18:39:06.967999 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 18:39:06.968137 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":105,"threads_started":1}
I20260927 18:39:06.968317 15382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:06.958118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:49193 (local address 127.11.251.254:32951)
0927 18:39:06.958209 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:06.958212 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:06.958489 (+ 277us) server_negotiation.cc:407] Connection header received
0927 18:39:06.959106 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:06.959108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:06.959119 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:06.959184 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:06.960278 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:06.960781 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:06.962093 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:06.962327 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:06.962409 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:06.964888 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:06.964890 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:06.964890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:06.964898 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:06.966491 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:06.967084 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:06.967086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:06.967086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:06.967153 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:06.967485 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:06.967486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:06.967486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:06.967672 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:06.967760 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:06.968053 (+ 293us) server_negotiation.cc:299] Negotiation successful
0927 18:39:06.968161 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260927 18:39:06.969086 15668 heartbeater.cc:347] Connected to a master server at 127.11.251.254:32951
I20260927 18:39:06.969226 15668 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:06.969501 15668 heartbeater.cc:511] Master 127.11.251.254:32951 requested a full tablet report, sending...
I20260927 18:39:06.970110 15342 ts_manager.cc:194] Registered new tserver with Master: 48d46dec14cf4ad3b1cdf398d1db4c78 (127.11.251.194:39807)
I20260927 18:39:06.970566 15342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:49193
WARNING: no policy specified for kudu/127.11.251.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
I20260927 18:39:06.992911 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:32951
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:37883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:07.083333 15673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:07.083571 15673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:07.083597 15673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:07.084856 15673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:07.084909 15673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:07.084939 15673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:39:07.086571 15673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:37883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:32951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:07.086861 15673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:07.087143 15673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:07.093055 15679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:07.093104 15681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:07.093104 15678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:07.093650 15673 server_base.cc:1034] running on GCE node
I20260927 18:39:07.093897 15673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:07.094166 15673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:07.095321 15673 hybrid_clock.cc:648] HybridClock initialized: now 1790534347095286 us; error 55 us; skew 500 ppm
Sep 27 18:39:07 dist-test-slave-z47j krb5kdc[15298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534347, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:07.098279 15673 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:39:07.098783 15673 webserver.cc:460] Webserver started at http://127.11.251.195:43687/ using document root <none> and password file <none>
I20260927 18:39:07.099009 15673 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:07.099054 15673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:07.099153 15673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:07.100348 15673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "07454d28aab749ce8ed6eb5cef975bf0"
format_stamp: "Formatted at 2026-09-27 18:39:07 on dist-test-slave-z47j"
server_key: "e74e3388d34b5554744099fff0bd9773"
server_key_iv: "5e2e3de5c351ec517149f50fea153164"
server_key_version: "encryptionkey@0"
I20260927 18:39:07.100693 15673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "07454d28aab749ce8ed6eb5cef975bf0"
format_stamp: "Formatted at 2026-09-27 18:39:07 on dist-test-slave-z47j"
server_key: "e74e3388d34b5554744099fff0bd9773"
server_key_iv: "5e2e3de5c351ec517149f50fea153164"
server_key_version: "encryptionkey@0"
I20260927 18:39:07.102336 15673 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:39:07.103255 15688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:07.103579 15673 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:07.103698 15673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "07454d28aab749ce8ed6eb5cef975bf0"
format_stamp: "Formatted at 2026-09-27 18:39:07 on dist-test-slave-z47j"
server_key: "e74e3388d34b5554744099fff0bd9773"
server_key_iv: "5e2e3de5c351ec517149f50fea153164"
server_key_version: "encryptionkey@0"
I20260927 18:39:07.103782 15673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:07.113205 15673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:07.115612 15673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:07.115805 15673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:07.116058 15673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:07.116387 15673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:07.116449 15673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:07.116492 15673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:07.116519 15673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:07.123808 15673 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:44503
I20260927 18:39:07.123863 15801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:44503 every 8 connection(s)
I20260927 18:39:07.124228 15673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:39:07.128604 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 15673
I20260927 18:39:07.128688 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:39:07.128918 12271 external_mini_cluster.cc:1428] Setting key cd6419a2f9617f7e5e6ab3d5da97bd59
Sep 27 18:39:07 dist-test-slave-z47j krb5kdc[15298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534347, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:07.135161 15382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:07.125133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:39429 (local address 127.11.251.254:32951)
0927 18:39:07.125209 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:07.125211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:07.125468 (+ 257us) server_negotiation.cc:407] Connection header received
0927 18:39:07.126044 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:07.126045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:07.126057 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:07.126135 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:07.127279 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:07.127890 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:07.129267 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:07.129615 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:07.129687 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:07.132200 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:07.132201 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:07.132202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:07.132210 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:07.133705 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:07.134145 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:07.134147 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:07.134147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:07.134205 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:07.134467 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:07.134468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:07.134468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:07.134590 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:07.134667 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:07.134968 (+ 301us) server_negotiation.cc:299] Negotiation successful
0927 18:39:07.135061 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 18:39:07.135382 15804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:07.125093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:32951 (local address 127.11.251.195:39429)
0927 18:39:07.125305 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:07.125353 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:07.125938 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:07.126221 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:07.126225 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:07.126903 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:07.127169 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:07.127175 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:07.128002 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:07.128002 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:07.129142 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:07.129145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:07.129744 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:07.129745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:07.129889 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:07.130500 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:07.130508 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:07.132094 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:07.133800 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:07.133807 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:07.133809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:07.134052 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:07.134280 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:07.134281 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:07.134281 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:07.134389 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:07.134656 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:07.134662 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:07.134816 (+ 154us) client_negotiation.cc:769] Sending connection context
0927 18:39:07.134937 (+ 121us) client_negotiation.cc:240] Negotiation successful
0927 18:39:07.135119 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":119,"threads_started":1}
I20260927 18:39:07.135929 15802 heartbeater.cc:347] Connected to a master server at 127.11.251.254:32951
I20260927 18:39:07.136082 15802 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:07.136309 15802 heartbeater.cc:511] Master 127.11.251.254:32951 requested a full tablet report, sending...
I20260927 18:39:07.136757 15342 ts_manager.cc:194] Registered new tserver with Master: 07454d28aab749ce8ed6eb5cef975bf0 (127.11.251.195:44503)
I20260927 18:39:07.137229 15342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:39429
I20260927 18:39:07.141153 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 18:39:07 dist-test-slave-z47j krb5kdc[15298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534347, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 27 18:39:07 dist-test-slave-z47j krb5kdc[15298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534347, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:07.166294 15382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:07.158661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34494 (local address 127.11.251.254:32951)
0927 18:39:07.158768 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:07.158770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:07.158873 (+ 103us) server_negotiation.cc:407] Connection header received
0927 18:39:07.158944 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:07.158945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:07.158956 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:07.159051 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:07.159650 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:07.160182 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:07.160992 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:07.161212 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:07.161273 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:07.163793 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:07.163795 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:07.163796 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:07.163803 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:07.165111 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:07.165474 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:07.165474 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:07.165475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:07.165526 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:07.165758 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:07.165758 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:07.165759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:07.165932 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:07.166020 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:07.166150 (+ 130us) server_negotiation.cc:299] Negotiation successful
0927 18:39:07.166218 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260927 18:39:07.166565 15342 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34494
I20260927 18:39:07.167351 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 15405
I20260927 18:39:07.172801 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 15539
I20260927 18:39:07.177690 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 15673
I20260927 18:39:07.182583 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 15311
2026-09-27T18:39:07Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1989 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:39:07 dist-test-slave-z47j krb5kdc[15814](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:39:07 dist-test-slave-z47j krb5kdc[15814](info): set up 2 sockets
Sep 27 18:39:07 dist-test-slave-z47j krb5kdc[15814](info): commencing operation
krb5kdc: starting...
W20260927 18:39:08.234037 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:39:08 dist-test-slave-z47j krb5kdc[15814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:39:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:39:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:39:08.360641 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:46699
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:38955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:46699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:08.452800 15827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:08.453032 15827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:08.453054 15827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:08.454421 15827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:08.454469 15827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:08.454484 15827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:08.454497 15827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:08.454509 15827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:39:08.456316 15827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:38955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:46699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:46699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:08.456584 15827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:08.456853 15827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:08.462504 15835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:08.462504 15832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:08.462522 15833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:08.462808 15827 server_base.cc:1034] running on GCE node
I20260927 18:39:08.463078 15827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:08.463397 15827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:08.464560 15827 hybrid_clock.cc:648] HybridClock initialized: now 1790534348464551 us; error 54 us; skew 500 ppm
Sep 27 18:39:08 dist-test-slave-z47j krb5kdc[15814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534348, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:08.467464 15827 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:39:08.467941 15827 webserver.cc:460] Webserver started at http://127.11.251.254:40297/ using document root <none> and password file <none>
I20260927 18:39:08.468204 15827 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:08.468274 15827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:08.468434 15827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:08.469695 15827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "2b56d241c64c4d16ad7874cc8b2e407e"
format_stamp: "Formatted at 2026-09-27 18:39:08 on dist-test-slave-z47j"
server_key: "85ed834b0ca3ef411d2dd2cbcbbedd58"
server_key_iv: "317c6a3b6516a65afc59c880d24c4222"
server_key_version: "encryptionkey@0"
I20260927 18:39:08.470096 15827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "2b56d241c64c4d16ad7874cc8b2e407e"
format_stamp: "Formatted at 2026-09-27 18:39:08 on dist-test-slave-z47j"
server_key: "85ed834b0ca3ef411d2dd2cbcbbedd58"
server_key_iv: "317c6a3b6516a65afc59c880d24c4222"
server_key_version: "encryptionkey@0"
I20260927 18:39:08.471578 15827 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 18:39:08.472489 15842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:08.472744 15827 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:39:08.472856 15827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "2b56d241c64c4d16ad7874cc8b2e407e"
format_stamp: "Formatted at 2026-09-27 18:39:08 on dist-test-slave-z47j"
server_key: "85ed834b0ca3ef411d2dd2cbcbbedd58"
server_key_iv: "317c6a3b6516a65afc59c880d24c4222"
server_key_version: "encryptionkey@0"
I20260927 18:39:08.472944 15827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:08.486999 15827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:08.489431 15827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:08.489681 15827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:08.494450 15827 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:46699
I20260927 18:39:08.494493 15894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:46699 every 8 connection(s)
I20260927 18:39:08.495056 15827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:39:08.495848 15895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:08.496693 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 15827
I20260927 18:39:08.496815 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:39:08.497014 12271 external_mini_cluster.cc:1428] Setting key afc7a9612689c56b3707f8e1e194f772
I20260927 18:39:08.499488 15895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e: Bootstrap starting.
I20260927 18:39:08.500389 15895 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:08.500814 15895 log.cc:824] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:08.501766 15895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e: No bootstrap required, opened a new log
I20260927 18:39:08.504770 15895 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b56d241c64c4d16ad7874cc8b2e407e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46699 } }
I20260927 18:39:08.504966 15895 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:08.505018 15895 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b56d241c64c4d16ad7874cc8b2e407e, State: Initialized, Role: FOLLOWER
Sep 27 18:39:08 dist-test-slave-z47j krb5kdc[15814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:08.505179 15895 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b56d241c64c4d16ad7874cc8b2e407e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46699 } }
I20260927 18:39:08.505283 15895 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:39:08.505321 15895 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:39:08.505393 15895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:08.506255 15895 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b56d241c64c4d16ad7874cc8b2e407e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46699 } }
I20260927 18:39:08.506431 15895 leader_election.cc:302] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b56d241c64c4d16ad7874cc8b2e407e; no voters:
I20260927 18:39:08.506655 15895 leader_election.cc:288] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:39:08.506704 15900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:08.506878 15900 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e [term 1 LEADER]: Becoming Leader. State: Replica: 2b56d241c64c4d16ad7874cc8b2e407e, State: Running, Role: LEADER
I20260927 18:39:08.506970 15900 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b56d241c64c4d16ad7874cc8b2e407e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46699 } }
I20260927 18:39:08.506992 15895 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:08.507310 15900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b56d241c64c4d16ad7874cc8b2e407e. Latest consensus state: current_term: 1 leader_uuid: "2b56d241c64c4d16ad7874cc8b2e407e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b56d241c64c4d16ad7874cc8b2e407e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46699 } } }
I20260927 18:39:08.507390 15900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:08.508739 15901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b56d241c64c4d16ad7874cc8b2e407e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b56d241c64c4d16ad7874cc8b2e407e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46699 } } }
I20260927 18:39:08.508972 15901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e [sys.catalog]: This master's current role is: LEADER
W20260927 18:39:08.509402 15916 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:39:08.509497 15916 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:39:08.509605 15904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:39:08.509801 15904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:39:08.510066 15898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:08.497633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48136 (local address 127.11.251.254:46699)
0927 18:39:08.497810 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:08.497818 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:08.497844 (+ 26us) server_negotiation.cc:407] Connection header received
0927 18:39:08.498294 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:08.498301 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:08.498347 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:08.498482 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:08.499489 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:08.500554 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:08.501790 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:08.502435 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:08.502535 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:08.505741 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:08.505744 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:08.505745 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:08.505755 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:08.508083 (+ 2328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:08.508480 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:08.508483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:08.508484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:08.508595 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:08.508831 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:08.508833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:08.508833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:08.509175 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:08.509344 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:08.509545 (+ 201us) server_negotiation.cc:299] Negotiation successful
0927 18:39:08.509731 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
I20260927 18:39:08.511478 15904 catalog_manager.cc:1347] Generated new cluster ID: e06a298ab4204870822e724f430180e5
I20260927 18:39:08.511535 15904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:39:08.532923 15904 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:39:08.533636 15904 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:39:08.538568 15904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b56d241c64c4d16ad7874cc8b2e407e: Generated new TSK 0
I20260927 18:39:08.538792 15904 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.251.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
I20260927 18:39:08.576483 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:46699
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:38955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:08.689884 15921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:08.690270 15921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:08.690301 15921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:08.691601 15921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:08.691653 15921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:08.691682 15921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:39:08.693367 15921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:38955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:46699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:08.693706 15921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:08.694119 15921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:08.699757 15926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:08.699790 15927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:08.699779 15929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:08.700045 15921 server_base.cc:1034] running on GCE node
I20260927 18:39:08.700284 15921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:08.700511 15921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:08.701664 15921 hybrid_clock.cc:648] HybridClock initialized: now 1790534348701643 us; error 36 us; skew 500 ppm
Sep 27 18:39:08 dist-test-slave-z47j krb5kdc[15814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534348, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:08.705120 15921 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:39:08.705746 15921 webserver.cc:460] Webserver started at http://127.11.251.193:41699/ using document root <none> and password file <none>
I20260927 18:39:08.706032 15921 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:08.706105 15921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:08.706266 15921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:08.707564 15921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "36146c2a8a1e4166af304f3135e3e8c3"
format_stamp: "Formatted at 2026-09-27 18:39:08 on dist-test-slave-z47j"
server_key: "b8d843ce92fd5ec367f9f0aaf8eb7a55"
server_key_iv: "a1a0858a8deb520328055cc456c663ae"
server_key_version: "encryptionkey@0"
I20260927 18:39:08.707952 15921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "36146c2a8a1e4166af304f3135e3e8c3"
format_stamp: "Formatted at 2026-09-27 18:39:08 on dist-test-slave-z47j"
server_key: "b8d843ce92fd5ec367f9f0aaf8eb7a55"
server_key_iv: "a1a0858a8deb520328055cc456c663ae"
server_key_version: "encryptionkey@0"
I20260927 18:39:08.709662 15921 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260927 18:39:08.710554 15936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:08.710807 15921 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:08.710922 15921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "36146c2a8a1e4166af304f3135e3e8c3"
format_stamp: "Formatted at 2026-09-27 18:39:08 on dist-test-slave-z47j"
server_key: "b8d843ce92fd5ec367f9f0aaf8eb7a55"
server_key_iv: "a1a0858a8deb520328055cc456c663ae"
server_key_version: "encryptionkey@0"
I20260927 18:39:08.711023 15921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:08.725042 15921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:08.727658 15921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:08.727872 15921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:08.728188 15921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:08.728631 15921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:08.728704 15921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:08.728775 15921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:08.728813 15921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:08.736359 15921 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:42439
I20260927 18:39:08.736393 16049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:42439 every 8 connection(s)
I20260927 18:39:08.736776 15921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:39:08.743676 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 15921
I20260927 18:39:08.743819 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:39:08.744083 12271 external_mini_cluster.cc:1428] Setting key 92f269e4b8d774e94dd3da80d2c1507f
Sep 27 18:39:08 dist-test-slave-z47j krb5kdc[15814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534348, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:08.748672 15898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:08.737704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:34009 (local address 127.11.251.254:46699)
0927 18:39:08.737786 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:08.737789 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:08.738102 (+ 313us) server_negotiation.cc:407] Connection header received
0927 18:39:08.739155 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:08.739157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:08.739168 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:08.739237 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:08.740052 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:08.740698 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:08.742052 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:08.742292 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:08.742356 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:08.745446 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:08.745448 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:08.745448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:08.745457 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:08.747095 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:08.747547 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:08.747549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:08.747549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:08.747621 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:08.747889 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:08.747890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:08.747890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:08.748068 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:08.748144 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:08.748512 (+ 368us) server_negotiation.cc:299] Negotiation successful
0927 18:39:08.748594 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:39:08.749024 16052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:08.737714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:46699 (local address 127.11.251.193:34009)
0927 18:39:08.737952 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:08.737997 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:08.739022 (+ 1025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:08.739335 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:08.739339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:08.739690 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:08.739919 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:08.739924 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:08.740790 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:08.740790 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:08.741969 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:08.741972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:08.742391 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:08.742392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:08.742526 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:08.743202 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:08.743213 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:08.745316 (+ 2103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:08.747188 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:08.747196 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:08.747198 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:08.747448 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:08.747698 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:08.747699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:08.747700 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:08.747810 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:08.748160 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:08.748167 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:08.748357 (+ 190us) client_negotiation.cc:769] Sending connection context
0927 18:39:08.748491 (+ 134us) client_negotiation.cc:240] Negotiation successful
0927 18:39:08.748699 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":104,"threads_started":1}
I20260927 18:39:08.749737 16050 heartbeater.cc:347] Connected to a master server at 127.11.251.254:46699
I20260927 18:39:08.749857 16050 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:08.750077 16050 heartbeater.cc:511] Master 127.11.251.254:46699 requested a full tablet report, sending...
I20260927 18:39:08.750702 15859 ts_manager.cc:194] Registered new tserver with Master: 36146c2a8a1e4166af304f3135e3e8c3 (127.11.251.193:42439)
I20260927 18:39:08.751367 15859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:34009
WARNING: no policy specified for kudu/127.11.251.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
I20260927 18:39:08.773063 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:46699
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:38955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:08.861142 16055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:08.861407 16055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:08.861445 16055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:08.862679 16055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:08.862751 16055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:08.862800 16055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:39:08.864362 16055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:38955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:46699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:08.864660 16055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:08.864908 16055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:08.870432 16061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:08.870496 16063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:08.870431 16060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:08.870721 16055 server_base.cc:1034] running on GCE node
I20260927 18:39:08.870957 16055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:08.871193 16055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:08.872390 16055 hybrid_clock.cc:648] HybridClock initialized: now 1790534348872381 us; error 37 us; skew 500 ppm
Sep 27 18:39:08 dist-test-slave-z47j krb5kdc[15814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534348, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:08.875314 16055 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:39:08.875867 16055 webserver.cc:460] Webserver started at http://127.11.251.194:42613/ using document root <none> and password file <none>
I20260927 18:39:08.876116 16055 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:08.876187 16055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:08.876345 16055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:08.877545 16055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "f87aa919fddf44ab864316cc3dd291b9"
format_stamp: "Formatted at 2026-09-27 18:39:08 on dist-test-slave-z47j"
server_key: "ddb9e65a805b686a6a4c7d3b8367a048"
server_key_iv: "1c630e1162d109663a83a274aec7287b"
server_key_version: "encryptionkey@0"
I20260927 18:39:08.877998 16055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "f87aa919fddf44ab864316cc3dd291b9"
format_stamp: "Formatted at 2026-09-27 18:39:08 on dist-test-slave-z47j"
server_key: "ddb9e65a805b686a6a4c7d3b8367a048"
server_key_iv: "1c630e1162d109663a83a274aec7287b"
server_key_version: "encryptionkey@0"
I20260927 18:39:08.879407 16055 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 18:39:08.880191 16070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:08.880362 16055 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:08.880473 16055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "f87aa919fddf44ab864316cc3dd291b9"
format_stamp: "Formatted at 2026-09-27 18:39:08 on dist-test-slave-z47j"
server_key: "ddb9e65a805b686a6a4c7d3b8367a048"
server_key_iv: "1c630e1162d109663a83a274aec7287b"
server_key_version: "encryptionkey@0"
I20260927 18:39:08.880548 16055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:08.897449 16055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:08.899600 16055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:08.899719 16055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:08.899935 16055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:08.900276 16055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:08.900310 16055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:08.900336 16055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:08.900348 16055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:08.907142 16055 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:43851
I20260927 18:39:08.907289 16183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:43851 every 8 connection(s)
I20260927 18:39:08.907613 16055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:39:08.908825 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 16055
I20260927 18:39:08.908970 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:39:08.909184 12271 external_mini_cluster.cc:1428] Setting key f793cc70aa71424040665711a94d8a62
Sep 27 18:39:08 dist-test-slave-z47j krb5kdc[15814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534348, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.251.195@KRBTEST.COM; defaulting to no policy
I20260927 18:39:08.919251 15898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:08.908494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:41147 (local address 127.11.251.254:46699)
0927 18:39:08.908573 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:08.908575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:08.908944 (+ 369us) server_negotiation.cc:407] Connection header received
0927 18:39:08.910359 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:08.910361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:08.910374 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:08.910425 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:08.911399 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:08.912022 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:08.913099 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:08.913320 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:08.913404 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:08.916272 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:08.916274 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:08.916274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:08.916282 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:08.917758 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:08.918239 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:08.918240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:08.918241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:08.918291 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:08.918575 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:08.918575 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:08.918576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:08.918685 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:08.918736 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:08.919100 (+ 364us) server_negotiation.cc:299] Negotiation successful
0927 18:39:08.919172 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:39:08.919538 16186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:08.908507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:46699 (local address 127.11.251.194:41147)
0927 18:39:08.908698 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:08.908751 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:08.910206 (+ 1455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:08.910517 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:08.910521 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:08.910967 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:08.911288 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:08.911293 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:08.912124 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:08.912125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:08.913002 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:08.913005 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:08.913408 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:08.913409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:08.913556 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:08.914179 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:08.914189 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:08.916157 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:08.917838 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:08.917845 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:08.917848 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:08.918159 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:08.918359 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:08.918360 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:08.918360 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:08.918500 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:08.918750 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:08.918758 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:08.918959 (+ 201us) client_negotiation.cc:769] Sending connection context
0927 18:39:08.919082 (+ 123us) client_negotiation.cc:240] Negotiation successful
0927 18:39:08.919266 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":102,"threads_started":1}
I20260927 18:39:08.920025 16184 heartbeater.cc:347] Connected to a master server at 127.11.251.254:46699
I20260927 18:39:08.920156 16184 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:08.920398 16184 heartbeater.cc:511] Master 127.11.251.254:46699 requested a full tablet report, sending...
I20260927 18:39:08.920869 15859 ts_manager.cc:194] Registered new tserver with Master: f87aa919fddf44ab864316cc3dd291b9 (127.11.251.194:43851)
I20260927 18:39:08.921386 15859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:41147
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
I20260927 18:39:08.937068 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:46699
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:38955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:09.027242 16189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:09.027485 16189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:09.027511 16189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:09.028649 16189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:09.028684 16189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:09.028712 16189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:39:09.030284 16189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:38955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:46699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:09.030503 16189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:09.030705 16189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:09.036394 16195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:09.036475 16197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:09.036516 16189 server_base.cc:1034] running on GCE node
W20260927 18:39:09.036440 16194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:09.036953 16189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:09.037251 16189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:09.038452 16189 hybrid_clock.cc:648] HybridClock initialized: now 1790534349038392 us; error 39 us; skew 500 ppm
Sep 27 18:39:09 dist-test-slave-z47j krb5kdc[15814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534349, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:09.041244 16189 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:39:09.041795 16189 webserver.cc:460] Webserver started at http://127.11.251.195:43927/ using document root <none> and password file <none>
I20260927 18:39:09.042042 16189 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:09.042089 16189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:09.042188 16189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:09.043282 16189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "c355511d63d940e3b37da4fa483fc7c9"
format_stamp: "Formatted at 2026-09-27 18:39:09 on dist-test-slave-z47j"
server_key: "fa0b85899c1b1dca4ba14bbbbcb1f03d"
server_key_iv: "7d94442832fe5ef33dda82416e4740b5"
server_key_version: "encryptionkey@0"
I20260927 18:39:09.043625 16189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "c355511d63d940e3b37da4fa483fc7c9"
format_stamp: "Formatted at 2026-09-27 18:39:09 on dist-test-slave-z47j"
server_key: "fa0b85899c1b1dca4ba14bbbbcb1f03d"
server_key_iv: "7d94442832fe5ef33dda82416e4740b5"
server_key_version: "encryptionkey@0"
I20260927 18:39:09.045058 16189 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:39:09.045826 16204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:09.046015 16189 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:09.046125 16189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "c355511d63d940e3b37da4fa483fc7c9"
format_stamp: "Formatted at 2026-09-27 18:39:09 on dist-test-slave-z47j"
server_key: "fa0b85899c1b1dca4ba14bbbbcb1f03d"
server_key_iv: "7d94442832fe5ef33dda82416e4740b5"
server_key_version: "encryptionkey@0"
I20260927 18:39:09.046200 16189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:09.062512 16189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:09.064733 16189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:09.064898 16189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:09.065182 16189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:09.065541 16189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:09.065644 16189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:09.065706 16189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:09.065737 16189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:09.072477 16189 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:38989
I20260927 18:39:09.072517 16317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:38989 every 8 connection(s)
I20260927 18:39:09.072891 16189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
Sep 27 18:39:09 dist-test-slave-z47j krb5kdc[15814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534349, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:09.082609 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 16189
I20260927 18:39:09.082710 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:39:09.082953 12271 external_mini_cluster.cc:1428] Setting key d021afa3b63137e0618b6191969bda17
I20260927 18:39:09.083498 15898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:09.073804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:39029 (local address 127.11.251.254:46699)
0927 18:39:09.073880 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:09.073888 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:09.074147 (+ 259us) server_negotiation.cc:407] Connection header received
0927 18:39:09.075217 (+ 1070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:09.075219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:09.075235 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:09.075301 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:09.076102 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:09.076659 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:09.077836 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:09.078041 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:09.078145 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:09.080505 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:09.080507 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:09.080507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:09.080515 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:09.081896 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:09.082265 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:09.082265 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:09.082266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:09.082317 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:09.082602 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:09.082602 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:09.082603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:09.082750 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:09.082836 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:09.083212 (+ 376us) server_negotiation.cc:299] Negotiation successful
0927 18:39:09.083336 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 18:39:09.083731 16320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:09.073820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:46699 (local address 127.11.251.195:39029)
0927 18:39:09.074010 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:09.074049 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:09.075091 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:09.075378 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:09.075382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:09.075695 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:09.075992 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:09.075997 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:09.076753 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:09.076754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:09.077740 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:09.077742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:09.078151 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:09.078153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:09.078294 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:09.078855 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:09.078863 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:09.080390 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:09.081973 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:09.081979 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:09.081981 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:09.082200 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:09.082373 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:09.082374 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:09.082374 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:09.082533 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:09.082863 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:09.082870 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:09.083051 (+ 181us) client_negotiation.cc:769] Sending connection context
0927 18:39:09.083180 (+ 129us) client_negotiation.cc:240] Negotiation successful
0927 18:39:09.083444 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":95,"threads_started":1}
I20260927 18:39:09.084160 16318 heartbeater.cc:347] Connected to a master server at 127.11.251.254:46699
I20260927 18:39:09.084306 16318 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:09.084553 16318 heartbeater.cc:511] Master 127.11.251.254:46699 requested a full tablet report, sending...
I20260927 18:39:09.084997 15859 ts_manager.cc:194] Registered new tserver with Master: c355511d63d940e3b37da4fa483fc7c9 (127.11.251.195:38989)
I20260927 18:39:09.085363 15859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:39029
I20260927 18:39:09.094940 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 18:39:09 dist-test-slave-z47j krb5kdc[15814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 18:39:09 dist-test-slave-z47j krb5kdc[15814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.193@KRBTEST.COM
I20260927 18:39:09.119808 16325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:09.111383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48094 (local address 127.11.251.193:42439)
0927 18:39:09.111586 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:09.111592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:09.111603 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:39:09.111636 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:09.111638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:09.111655 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:09.111741 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:09.112314 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:09.113103 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:09.114042 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:09.114389 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:09.114456 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:09.116831 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:09.116839 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:09.116839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:09.116847 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:09.118468 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:09.118785 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:09.118788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:09.118788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:09.118872 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:09.119090 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:09.119091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:09.119091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:09.119234 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:09.119381 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:09.119427 (+ 46us) server_negotiation.cc:299] Negotiation successful
0927 18:39:09.119511 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":114,"threads_started":1}
Sep 27 18:39:09 dist-test-slave-z47j krb5kdc[15814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:09.129259 15898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:09.121445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48142 (local address 127.11.251.254:46699)
0927 18:39:09.121527 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:09.121529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:09.121544 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:39:09.121704 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:09.121706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:09.121718 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:09.121769 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:09.122321 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:09.122925 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:09.123681 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:09.123911 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:09.124002 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:09.126518 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:09.126520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:09.126520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:09.126527 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:09.127803 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:09.128181 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:09.128183 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:09.128184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:09.128245 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:09.128460 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:09.128461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:09.128462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:09.128641 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:09.128763 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:09.128921 (+ 158us) server_negotiation.cc:299] Negotiation successful
0927 18:39:09.129135 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260927 18:39:09.129628 15859 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48142
I20260927 18:39:09.130076 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 15921
I20260927 18:39:09.135967 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 16055
I20260927 18:39:09.141256 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 16189
I20260927 18:39:09.145912 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 15827
2026-09-27T18:39:09Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1963 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:39:09 dist-test-slave-z47j krb5kdc[16330](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:39:09 dist-test-slave-z47j krb5kdc[16330](info): set up 2 sockets
Sep 27 18:39:09 dist-test-slave-z47j krb5kdc[16330](info): commencing operation
krb5kdc: starting...
W20260927 18:39:10.223098 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260927 18:39:10.538462 12271 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.315s user 0.001s sys 0.001s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260927 18:39:10.847730 12271 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.309s user 0.001s sys 0.001s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260927 18:39:11.148813 12271 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.301s user 0.001s sys 0.001s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/test-user.keytab.
W20260927 18:39:11.916690 12271 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.768s user 0.000s sys 0.002s
Sep 27 18:39:11 dist-test-slave-z47j krb5kdc[16330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:39:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:39:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
W20260927 18:39:13.192173 12271 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.11.251.254: real 1.250s user 0.001s sys 0.002s
I20260927 18:39:13.192651 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:41063
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:36285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:41063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:13.276324 16343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:13.276583 16343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:13.276623 16343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:13.277870 16343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:13.277935 16343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:13.277963 16343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:13.277995 16343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:13.278021 16343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:39:13.279539 16343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:36285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:41063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:41063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:13.279820 16343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:13.280047 16343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:13.285686 16348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:13.285674 16349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:13.285806 16343 server_base.cc:1034] running on GCE node
W20260927 18:39:13.285845 16351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:13.286381 16343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:13.286742 16343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:13.287921 16343 hybrid_clock.cc:648] HybridClock initialized: now 1790534353287888 us; error 52 us; skew 500 ppm
Sep 27 18:39:13 dist-test-slave-z47j krb5kdc[16330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534353, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:13.299119 16343 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:39:13.299644 16343 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:13.299729 16343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:13.299899 16343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:13.301502 16343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "91977009379546bca6eb5534bbd2a214"
format_stamp: "Formatted at 2026-09-27 18:39:13 on dist-test-slave-z47j"
server_key: "f2ea70009e9b7d077f2cb873598429f3"
server_key_iv: "1b3cea5ebadd39e1c8e80af1b1a92b9b"
server_key_version: "encryptionkey@0"
I20260927 18:39:13.302065 16343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "91977009379546bca6eb5534bbd2a214"
format_stamp: "Formatted at 2026-09-27 18:39:13 on dist-test-slave-z47j"
server_key: "f2ea70009e9b7d077f2cb873598429f3"
server_key_iv: "1b3cea5ebadd39e1c8e80af1b1a92b9b"
server_key_version: "encryptionkey@0"
I20260927 18:39:13.303779 16343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:39:13.304674 16357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:13.304919 16343 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:13.305030 16343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "91977009379546bca6eb5534bbd2a214"
format_stamp: "Formatted at 2026-09-27 18:39:13 on dist-test-slave-z47j"
server_key: "f2ea70009e9b7d077f2cb873598429f3"
server_key_iv: "1b3cea5ebadd39e1c8e80af1b1a92b9b"
server_key_version: "encryptionkey@0"
I20260927 18:39:13.305153 16343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:13.317768 16343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:13.319653 16343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:13.319815 16343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:13.324196 16343 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:41063
I20260927 18:39:13.324246 16409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:41063 every 8 connection(s)
I20260927 18:39:13.324553 16343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:39:13.325773 16410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:13.328428 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 16343
I20260927 18:39:13.328528 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:39:13.328763 12271 external_mini_cluster.cc:1428] Setting key d8c05a2ab4b1572d5506925973ae03d9
I20260927 18:39:13.329093 16410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214: Bootstrap starting.
I20260927 18:39:13.329991 16410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:13.330391 16410 log.cc:824] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:13.331192 16410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214: No bootstrap required, opened a new log
I20260927 18:39:13.333388 16410 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91977009379546bca6eb5534bbd2a214" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41063 } }
I20260927 18:39:13.333604 16410 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:13.333679 16410 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91977009379546bca6eb5534bbd2a214, State: Initialized, Role: FOLLOWER
I20260927 18:39:13.333832 16410 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91977009379546bca6eb5534bbd2a214" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41063 } }
I20260927 18:39:13.333951 16410 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:39:13.333987 16410 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:39:13.334036 16410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214 [term 0 FOLLOWER]: Advancing to term 1
Sep 27 18:39:13 dist-test-slave-z47j krb5kdc[16330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:13.340171 16412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:13.329414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45854 (local address 127.11.251.254:41063)
0927 18:39:13.329625 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:13.329635 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:13.330613 (+ 978us) server_negotiation.cc:407] Connection header received
0927 18:39:13.331281 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:13.331313 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:13.331387 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:13.331535 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:13.332162 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:13.333050 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:13.333967 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:13.334478 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:13.334610 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:13.337133 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:13.337136 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:13.337137 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:13.337147 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:13.338696 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:13.339066 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:13.339068 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:13.339069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:13.339163 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:13.339371 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:13.339372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:13.339373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:13.339600 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:13.339746 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:13.339847 (+ 101us) server_negotiation.cc:299] Negotiation successful
0927 18:39:13.339980 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
I20260927 18:39:13.383893 16410 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91977009379546bca6eb5534bbd2a214" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41063 } }
I20260927 18:39:13.384194 16410 leader_election.cc:302] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91977009379546bca6eb5534bbd2a214; no voters:
I20260927 18:39:13.384497 16410 leader_election.cc:288] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:39:13.384546 16415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:13.384698 16415 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214 [term 1 LEADER]: Becoming Leader. State: Replica: 91977009379546bca6eb5534bbd2a214, State: Running, Role: LEADER
I20260927 18:39:13.384915 16410 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:13.384927 16415 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91977009379546bca6eb5534bbd2a214" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41063 } }
I20260927 18:39:13.385566 16416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91977009379546bca6eb5534bbd2a214. Latest consensus state: current_term: 1 leader_uuid: "91977009379546bca6eb5534bbd2a214" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91977009379546bca6eb5534bbd2a214" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41063 } } }
I20260927 18:39:13.385803 16416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:13.385640 16417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91977009379546bca6eb5534bbd2a214" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91977009379546bca6eb5534bbd2a214" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41063 } } }
I20260927 18:39:13.385957 16417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:13.386473 16424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:39:13.387184 16424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:39:13.389659 16424 catalog_manager.cc:1347] Generated new cluster ID: 8377ea8b008845c39f12274f244ab738
I20260927 18:39:13.389719 16424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:39:13.397851 16424 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:39:13.398463 16424 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:39:13.409538 16424 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214: Generated new TSK 0
I20260927 18:39:13.409806 16424 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.251.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
W20260927 18:39:13.874291 12271 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.11.251.193: real 0.460s user 0.000s sys 0.003s
I20260927 18:39:13.874712 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41063
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:36285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:13.966169 16436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:13.966419 16436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:13.966450 16436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:13.967712 16436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:13.967752 16436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:13.967780 16436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:39:13.969370 16436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:36285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:13.969699 16436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:13.969964 16436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:13.975582 16444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:13.975629 16441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:13.975708 16442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:13.975863 16436 server_base.cc:1034] running on GCE node
I20260927 18:39:13.976178 16436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:13.976459 16436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:13.977670 16436 hybrid_clock.cc:648] HybridClock initialized: now 1790534353977647 us; error 41 us; skew 500 ppm
Sep 27 18:39:13 dist-test-slave-z47j krb5kdc[16330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534353, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:13.980628 16436 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:39:13.981096 16436 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:13.981176 16436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:13.981343 16436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:13.982640 16436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "fa56c467fcda49a98d8057fa2c53cc4b"
format_stamp: "Formatted at 2026-09-27 18:39:13 on dist-test-slave-z47j"
server_key: "224b4af96e9116f29c120490393eacc4"
server_key_iv: "865953e62a0cb403c478c4cd203dc2c0"
server_key_version: "encryptionkey@0"
I20260927 18:39:13.983048 16436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "fa56c467fcda49a98d8057fa2c53cc4b"
format_stamp: "Formatted at 2026-09-27 18:39:13 on dist-test-slave-z47j"
server_key: "224b4af96e9116f29c120490393eacc4"
server_key_iv: "865953e62a0cb403c478c4cd203dc2c0"
server_key_version: "encryptionkey@0"
I20260927 18:39:13.984527 16436 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:39:13.985384 16450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:13.985620 16436 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:13.985729 16436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "fa56c467fcda49a98d8057fa2c53cc4b"
format_stamp: "Formatted at 2026-09-27 18:39:13 on dist-test-slave-z47j"
server_key: "224b4af96e9116f29c120490393eacc4"
server_key_iv: "865953e62a0cb403c478c4cd203dc2c0"
server_key_version: "encryptionkey@0"
I20260927 18:39:13.985814 16436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:13.999718 16436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:14.001883 16436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:14.002053 16436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:14.002308 16436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:14.002669 16436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:14.002732 16436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:14.002792 16436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:14.002822 16436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:14.010411 16436 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:43389
I20260927 18:39:14.010437 16563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:43389 every 8 connection(s)
I20260927 18:39:14.010792 16436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
Sep 27 18:39:14 dist-test-slave-z47j krb5kdc[16330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534353, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:14.020715 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 16436
I20260927 18:39:14.020864 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:39:14.021070 12271 external_mini_cluster.cc:1428] Setting key 086160d344bb3cd8b6382eba131486ee
I20260927 18:39:14.021495 16566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.011650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:55267 (local address 127.11.251.254:41063)
0927 18:39:14.011798 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:14.011800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:14.011986 (+ 186us) server_negotiation.cc:407] Connection header received
0927 18:39:14.012550 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:14.012551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:14.012565 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:14.012636 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:14.013710 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.014275 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.015308 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.015536 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.015595 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:14.018083 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:14.018084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:14.018085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:14.018103 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:14.019665 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:14.020149 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:14.020150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:14.020151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:14.020246 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:14.020577 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:14.020579 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:14.020579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:14.020772 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:14.020880 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:14.021275 (+ 395us) server_negotiation.cc:299] Negotiation successful
0927 18:39:14.021382 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":57,"threads_started":1}
I20260927 18:39:14.021728 16567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.011639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41063 (local address 127.11.251.193:55267)
0927 18:39:14.011844 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:14.011885 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:14.012458 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:14.012701 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:14.012705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:14.013015 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:14.013622 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:14.013627 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.014357 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.014357 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:14.015226 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:14.015228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.015630 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.015631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:14.015744 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:14.016379 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:14.016388 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:14.017968 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:14.019763 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:14.019770 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:14.019773 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:14.020028 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:14.020320 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:14.020320 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:14.020321 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:14.020472 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:14.020884 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:14.020893 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:14.021068 (+ 175us) client_negotiation.cc:769] Sending connection context
0927 18:39:14.021236 (+ 168us) client_negotiation.cc:240] Negotiation successful
0927 18:39:14.021430 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":101,"threads_started":1}
I20260927 18:39:14.022346 16564 heartbeater.cc:347] Connected to a master server at 127.11.251.254:41063
I20260927 18:39:14.022472 16564 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:14.022675 16564 heartbeater.cc:511] Master 127.11.251.254:41063 requested a full tablet report, sending...
I20260927 18:39:14.023278 16374 ts_manager.cc:194] Registered new tserver with Master: fa56c467fcda49a98d8057fa2c53cc4b (127.11.251.193:43389)
I20260927 18:39:14.023938 16374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:55267
WARNING: no policy specified for kudu/127.11.251.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
I20260927 18:39:14.048627 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41063
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:36285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:14.139125 16570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:14.139400 16570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:14.139437 16570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:14.140653 16570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:14.140718 16570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:14.140764 16570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:39:14.142421 16570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:36285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:14.142722 16570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:14.142976 16570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:14.148427 16575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:14.148411 16576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:14.148504 16570 server_base.cc:1034] running on GCE node
W20260927 18:39:14.148411 16578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:14.149022 16570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:14.149302 16570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:14.150462 16570 hybrid_clock.cc:648] HybridClock initialized: now 1790534354150452 us; error 31 us; skew 500 ppm
Sep 27 18:39:14 dist-test-slave-z47j krb5kdc[16330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534354, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:14.153126 16570 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:39:14.153534 16570 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:14.153631 16570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:14.153789 16570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:14.154981 16570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "e2cdf0d8e2884a83b6a642a3b25681e8"
format_stamp: "Formatted at 2026-09-27 18:39:14 on dist-test-slave-z47j"
server_key: "e540ec958d318a557b0abd71d37dd9db"
server_key_iv: "da586255669f22aeb34d6a707506ef66"
server_key_version: "encryptionkey@0"
I20260927 18:39:14.155385 16570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "e2cdf0d8e2884a83b6a642a3b25681e8"
format_stamp: "Formatted at 2026-09-27 18:39:14 on dist-test-slave-z47j"
server_key: "e540ec958d318a557b0abd71d37dd9db"
server_key_iv: "da586255669f22aeb34d6a707506ef66"
server_key_version: "encryptionkey@0"
I20260927 18:39:14.156862 16570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:39:14.157716 16584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:14.157898 16570 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:14.158000 16570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "e2cdf0d8e2884a83b6a642a3b25681e8"
format_stamp: "Formatted at 2026-09-27 18:39:14 on dist-test-slave-z47j"
server_key: "e540ec958d318a557b0abd71d37dd9db"
server_key_iv: "da586255669f22aeb34d6a707506ef66"
server_key_version: "encryptionkey@0"
I20260927 18:39:14.158093 16570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:14.179411 16570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:14.181651 16570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:14.181826 16570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:14.182108 16570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:14.182489 16570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:14.182554 16570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:14.182592 16570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:14.182641 16570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:14.189867 16570 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:41855
I20260927 18:39:14.189911 16697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:41855 every 8 connection(s)
I20260927 18:39:14.190233 16570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:39:14.194083 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 16570
I20260927 18:39:14.194185 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:39:14.194396 12271 external_mini_cluster.cc:1428] Setting key cf6ac6bfa71ba07f5120975bf957f3f1
Sep 27 18:39:14 dist-test-slave-z47j krb5kdc[16330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534354, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:14.201113 16566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.191043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:59615 (local address 127.11.251.254:41063)
0927 18:39:14.191122 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:14.191125 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:14.191321 (+ 196us) server_negotiation.cc:407] Connection header received
0927 18:39:14.191927 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:14.191928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:14.191941 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:14.192006 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:14.193000 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.193560 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.194689 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.194985 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.195059 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:14.197831 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:14.197833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:14.197833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:14.197840 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:14.199551 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:14.199980 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:14.199982 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:14.199983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:14.200054 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:14.200302 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:14.200303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:14.200303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:14.200476 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:14.200551 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:14.200853 (+ 302us) server_negotiation.cc:299] Negotiation successful
0927 18:39:14.200973 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 18:39:14.201134 16700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.191016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41063 (local address 127.11.251.194:59615)
0927 18:39:14.191178 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:14.191224 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:14.191819 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:14.192065 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:14.192074 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:14.192369 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:14.192919 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:14.192923 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.193648 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.193649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:14.194623 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:14.194626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.195088 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.195089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:14.195233 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:14.195825 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:14.195833 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:14.197725 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:14.199651 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:14.199657 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:14.199660 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:14.199901 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:14.200129 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:14.200130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:14.200130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:14.200233 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:14.200549 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:14.200554 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:14.200709 (+ 155us) client_negotiation.cc:769] Sending connection context
0927 18:39:14.200797 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 18:39:14.200936 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
I20260927 18:39:14.201807 16698 heartbeater.cc:347] Connected to a master server at 127.11.251.254:41063
I20260927 18:39:14.201913 16698 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:14.202116 16698 heartbeater.cc:511] Master 127.11.251.254:41063 requested a full tablet report, sending...
I20260927 18:39:14.202553 16374 ts_manager.cc:194] Registered new tserver with Master: e2cdf0d8e2884a83b6a642a3b25681e8 (127.11.251.194:41855)
I20260927 18:39:14.202965 16374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:59615
WARNING: no policy specified for kudu/127.11.251.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
I20260927 18:39:14.222573 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41063
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:36285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:14.313369 16703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:14.313668 16703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:14.313711 16703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:14.314961 16703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:14.315032 16703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:14.315090 16703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:39:14.316762 16703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:36285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:14.317067 16703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:14.317333 16703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:14.322732 16708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:14.322918 16703 server_base.cc:1034] running on GCE node
W20260927 18:39:14.322933 16709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:14.323047 16711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:14.323303 16703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:14.323596 16703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:14.324816 16703 hybrid_clock.cc:648] HybridClock initialized: now 1790534354324789 us; error 52 us; skew 500 ppm
Sep 27 18:39:14 dist-test-slave-z47j krb5kdc[16330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534354, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:14.328059 16703 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:39:14.328521 16703 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:14.328572 16703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:14.328754 16703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:14.329998 16703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "e1a6fe3acd834ed5a893cffea0b7dc22"
format_stamp: "Formatted at 2026-09-27 18:39:14 on dist-test-slave-z47j"
server_key: "9e88aae4dcfe972dcf42c9d05ed4510f"
server_key_iv: "573b707075e7ff90f049be32957b3486"
server_key_version: "encryptionkey@0"
I20260927 18:39:14.330408 16703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "e1a6fe3acd834ed5a893cffea0b7dc22"
format_stamp: "Formatted at 2026-09-27 18:39:14 on dist-test-slave-z47j"
server_key: "9e88aae4dcfe972dcf42c9d05ed4510f"
server_key_iv: "573b707075e7ff90f049be32957b3486"
server_key_version: "encryptionkey@0"
I20260927 18:39:14.331853 16703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:39:14.332701 16717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:14.333012 16703 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:14.333139 16703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "e1a6fe3acd834ed5a893cffea0b7dc22"
format_stamp: "Formatted at 2026-09-27 18:39:14 on dist-test-slave-z47j"
server_key: "9e88aae4dcfe972dcf42c9d05ed4510f"
server_key_iv: "573b707075e7ff90f049be32957b3486"
server_key_version: "encryptionkey@0"
I20260927 18:39:14.333215 16703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:14.350805 16703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:14.352985 16703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:14.353164 16703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:14.353453 16703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:14.353871 16703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:14.353929 16703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:14.353967 16703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:14.353996 16703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:14.361146 16703 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:34683
I20260927 18:39:14.361207 16830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:34683 every 8 connection(s)
I20260927 18:39:14.361531 16703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
Sep 27 18:39:14 dist-test-slave-z47j krb5kdc[16330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534354, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:14.368542 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 16703
I20260927 18:39:14.368706 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:39:14.368942 12271 external_mini_cluster.cc:1428] Setting key b4a280cef6d4bd07e568e3fa74fe7b25
I20260927 18:39:14.372095 16566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.362509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:55843 (local address 127.11.251.254:41063)
0927 18:39:14.362590 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:14.362593 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:14.362851 (+ 258us) server_negotiation.cc:407] Connection header received
0927 18:39:14.363401 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:14.363402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:14.363414 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:14.363472 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:14.364550 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.365130 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.366233 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.366423 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.366478 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:14.369065 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:14.369067 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:14.369068 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:14.369082 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:14.370645 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:14.371076 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:14.371077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:14.371078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:14.371136 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:14.371398 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:14.371399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:14.371399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:14.371528 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:14.371622 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:14.371901 (+ 279us) server_negotiation.cc:299] Negotiation successful
0927 18:39:14.371994 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:39:14.372314 16833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.362500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41063 (local address 127.11.251.195:55843)
0927 18:39:14.362711 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:14.362753 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:14.363311 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:14.363559 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:14.363564 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:14.363869 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:14.364458 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:14.364463 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.365224 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.365225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:14.366159 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:14.366162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.366529 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.366530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:14.366692 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:14.367247 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:14.367256 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:14.368965 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:14.370741 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:14.370748 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:14.370751 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:14.370984 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:14.371209 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:14.371209 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:14.371210 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:14.371331 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:14.371610 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:14.371616 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:14.371766 (+ 150us) client_negotiation.cc:769] Sending connection context
0927 18:39:14.371879 (+ 113us) client_negotiation.cc:240] Negotiation successful
0927 18:39:14.372051 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":99,"threads_started":1}
I20260927 18:39:14.372828 16831 heartbeater.cc:347] Connected to a master server at 127.11.251.254:41063
I20260927 18:39:14.372920 16831 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:14.373056 16831 heartbeater.cc:511] Master 127.11.251.254:41063 requested a full tablet report, sending...
I20260927 18:39:14.373399 16374 ts_manager.cc:194] Registered new tserver with Master: e1a6fe3acd834ed5a893cffea0b7dc22 (127.11.251.195:34683)
I20260927 18:39:14.373847 16374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:55843
I20260927 18:39:14.381346 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:39:14.389170 16566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.382951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45860 (local address 127.11.251.254:41063)
0927 18:39:14.383063 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:14.383065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:14.383117 (+ 52us) server_negotiation.cc:407] Connection header received
0927 18:39:14.383163 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:14.383163 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:14.383174 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:14.383231 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:14.383930 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.384457 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.385238 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.385484 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.385544 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:14.386876 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:14.386877 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:14.386878 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:14.386884 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:14.388202 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:14.388489 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:14.388490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:14.388490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:14.388554 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:14.388700 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:14.388700 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:14.388700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:14.388847 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:14.388926 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:14.388979 (+ 53us) server_negotiation.cc:299] Negotiation successful
0927 18:39:14.389054 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260927 18:39:14.391268 16374 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:39:14.393349 16374 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:39:14.401185 16842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.396309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:43389 (local address 127.0.0.1:57274)
0927 18:39:14.397044 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:14.397053 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:14.397106 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:14.397312 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:14.397312 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:14.397314 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:14.397373 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:14.397375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.398794 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.398795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:14.400132 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:14.400134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.400851 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.400852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:14.400934 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:14.400950 (+ 16us) client_negotiation.cc:769] Sending connection context
0927 18:39:14.401020 (+ 70us) client_negotiation.cc:240] Negotiation successful
0927 18:39:14.401081 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"mutex_wait_us":52,"thread_start_us":108,"threads_started":1}
I20260927 18:39:14.401635 16846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.396592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57274 (local address 127.11.251.193:43389)
0927 18:39:14.396873 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:14.396878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:14.397166 (+ 288us) server_negotiation.cc:407] Connection header received
0927 18:39:14.397214 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:14.397217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:14.397233 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:14.397315 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:14.397435 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.398673 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.400223 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.400749 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.400845 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:14.400913 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:14.401421 (+ 508us) server_negotiation.cc:299] Negotiation successful
0927 18:39:14.401519 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":100,"threads_started":1}
I20260927 18:39:14.402432 16498 tablet_service.cc:1467] Processing CreateTablet for tablet b43f63c9f4c34c55834cd5092ed4ea6f (DEFAULT_TABLE table=test-table [id=8a60fffd054f48cebf987704fc6f5d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:14.402853 16498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b43f63c9f4c34c55834cd5092ed4ea6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:14.404947 16844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.396308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:34683 (local address 127.0.0.1:59456)
0927 18:39:14.396783 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:14.396824 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:14.396920 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:14.397307 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:14.397308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:14.397311 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:14.397373 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:14.397376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.398630 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.398631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:14.400131 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:14.400134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.404642 (+ 4508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.404643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:14.404724 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:14.404729 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:39:14.404790 (+ 61us) client_negotiation.cc:240] Negotiation successful
0927 18:39:14.404844 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":30,"thread_start_us":134,"threads_started":1}
I20260927 18:39:14.405093 16843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.396310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:41855 (local address 127.0.0.1:39278)
0927 18:39:14.396811 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:14.396828 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:14.396912 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:14.397249 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:14.397254 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:14.397266 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:14.397366 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:14.397369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.400249 (+ 2880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.400250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:14.400957 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:14.400958 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.404764 (+ 3806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.404765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:14.404856 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:14.404861 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:39:14.404932 (+ 71us) client_negotiation.cc:240] Negotiation successful
0927 18:39:14.404989 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":2,"thread_start_us":163,"threads_started":1}
I20260927 18:39:14.405232 16845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.396571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59456 (local address 127.11.251.195:34683)
0927 18:39:14.396745 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:14.396751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:14.396912 (+ 161us) server_negotiation.cc:407] Connection header received
0927 18:39:14.397099 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:14.397102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:14.397121 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:14.397193 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:14.397695 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.398491 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.400960 (+ 2469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.401806 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.401883 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:14.401933 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:14.405069 (+ 3136us) server_negotiation.cc:299] Negotiation successful
0927 18:39:14.405136 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
I20260927 18:39:14.405414 16847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.396645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39278 (local address 127.11.251.194:41855)
0927 18:39:14.396952 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:14.396956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:14.396970 (+ 14us) server_negotiation.cc:407] Connection header received
0927 18:39:14.397026 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:14.397029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:14.397047 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:14.397136 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:14.397463 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.398677 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.401060 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.401874 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.401951 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:14.402009 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:14.405247 (+ 3238us) server_negotiation.cc:299] Negotiation successful
0927 18:39:14.405311 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":90,"threads_started":1}
I20260927 18:39:14.406073 16765 tablet_service.cc:1467] Processing CreateTablet for tablet b43f63c9f4c34c55834cd5092ed4ea6f (DEFAULT_TABLE table=test-table [id=8a60fffd054f48cebf987704fc6f5d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:14.406343 16848 tablet_bootstrap.cc:492] T b43f63c9f4c34c55834cd5092ed4ea6f P fa56c467fcda49a98d8057fa2c53cc4b: Bootstrap starting.
I20260927 18:39:14.406303 16632 tablet_service.cc:1467] Processing CreateTablet for tablet b43f63c9f4c34c55834cd5092ed4ea6f (DEFAULT_TABLE table=test-table [id=8a60fffd054f48cebf987704fc6f5d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:14.406486 16765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b43f63c9f4c34c55834cd5092ed4ea6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:14.406631 16632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b43f63c9f4c34c55834cd5092ed4ea6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:14.407027 16848 tablet_bootstrap.cc:654] T b43f63c9f4c34c55834cd5092ed4ea6f P fa56c467fcda49a98d8057fa2c53cc4b: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:14.407619 16848 log.cc:824] T b43f63c9f4c34c55834cd5092ed4ea6f P fa56c467fcda49a98d8057fa2c53cc4b: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:14.408537 16848 tablet_bootstrap.cc:492] T b43f63c9f4c34c55834cd5092ed4ea6f P fa56c467fcda49a98d8057fa2c53cc4b: No bootstrap required, opened a new log
I20260927 18:39:14.408591 16848 ts_tablet_manager.cc:1397] T b43f63c9f4c34c55834cd5092ed4ea6f P fa56c467fcda49a98d8057fa2c53cc4b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:39:14.410475 16850 tablet_bootstrap.cc:492] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8: Bootstrap starting.
I20260927 18:39:14.410532 16851 tablet_bootstrap.cc:492] T b43f63c9f4c34c55834cd5092ed4ea6f P e1a6fe3acd834ed5a893cffea0b7dc22: Bootstrap starting.
I20260927 18:39:14.410662 16848 raft_consensus.cc:348] T b43f63c9f4c34c55834cd5092ed4ea6f P fa56c467fcda49a98d8057fa2c53cc4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2cdf0d8e2884a83b6a642a3b25681e8" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41855 } } peers { permanent_uuid: "fa56c467fcda49a98d8057fa2c53cc4b" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43389 } } peers { permanent_uuid: "e1a6fe3acd834ed5a893cffea0b7dc22" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 34683 } }
I20260927 18:39:14.410842 16848 raft_consensus.cc:374] T b43f63c9f4c34c55834cd5092ed4ea6f P fa56c467fcda49a98d8057fa2c53cc4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:14.410905 16848 raft_consensus.cc:729] T b43f63c9f4c34c55834cd5092ed4ea6f P fa56c467fcda49a98d8057fa2c53cc4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa56c467fcda49a98d8057fa2c53cc4b, State: Initialized, Role: FOLLOWER
I20260927 18:39:14.411067 16848 consensus_queue.cc:260] T b43f63c9f4c34c55834cd5092ed4ea6f P fa56c467fcda49a98d8057fa2c53cc4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2cdf0d8e2884a83b6a642a3b25681e8" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41855 } } peers { permanent_uuid: "fa56c467fcda49a98d8057fa2c53cc4b" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43389 } } peers { permanent_uuid: "e1a6fe3acd834ed5a893cffea0b7dc22" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 34683 } }
I20260927 18:39:14.411229 16850 tablet_bootstrap.cc:654] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:14.411229 16851 tablet_bootstrap.cc:654] T b43f63c9f4c34c55834cd5092ed4ea6f P e1a6fe3acd834ed5a893cffea0b7dc22: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:14.411394 16848 ts_tablet_manager.cc:1428] T b43f63c9f4c34c55834cd5092ed4ea6f P fa56c467fcda49a98d8057fa2c53cc4b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:39:14.411422 16564 heartbeater.cc:503] Master 127.11.251.254:41063 was elected leader, sending a full tablet report...
I20260927 18:39:14.412002 16851 log.cc:824] T b43f63c9f4c34c55834cd5092ed4ea6f P e1a6fe3acd834ed5a893cffea0b7dc22: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:14.412002 16850 log.cc:824] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:14.412789 16850 tablet_bootstrap.cc:492] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8: No bootstrap required, opened a new log
I20260927 18:39:14.412853 16850 ts_tablet_manager.cc:1397] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:39:14.413960 16851 tablet_bootstrap.cc:492] T b43f63c9f4c34c55834cd5092ed4ea6f P e1a6fe3acd834ed5a893cffea0b7dc22: No bootstrap required, opened a new log
I20260927 18:39:14.414038 16851 ts_tablet_manager.cc:1397] T b43f63c9f4c34c55834cd5092ed4ea6f P e1a6fe3acd834ed5a893cffea0b7dc22: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260927 18:39:14.414842 16850 raft_consensus.cc:348] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2cdf0d8e2884a83b6a642a3b25681e8" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41855 } } peers { permanent_uuid: "fa56c467fcda49a98d8057fa2c53cc4b" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43389 } } peers { permanent_uuid: "e1a6fe3acd834ed5a893cffea0b7dc22" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 34683 } }
I20260927 18:39:14.415030 16850 raft_consensus.cc:374] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:14.415066 16850 raft_consensus.cc:729] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2cdf0d8e2884a83b6a642a3b25681e8, State: Initialized, Role: FOLLOWER
I20260927 18:39:14.415186 16850 consensus_queue.cc:260] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2cdf0d8e2884a83b6a642a3b25681e8" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41855 } } peers { permanent_uuid: "fa56c467fcda49a98d8057fa2c53cc4b" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43389 } } peers { permanent_uuid: "e1a6fe3acd834ed5a893cffea0b7dc22" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 34683 } }
I20260927 18:39:14.415484 16850 ts_tablet_manager.cc:1428] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:39:14.415506 16698 heartbeater.cc:503] Master 127.11.251.254:41063 was elected leader, sending a full tablet report...
I20260927 18:39:14.416288 16851 raft_consensus.cc:348] T b43f63c9f4c34c55834cd5092ed4ea6f P e1a6fe3acd834ed5a893cffea0b7dc22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2cdf0d8e2884a83b6a642a3b25681e8" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41855 } } peers { permanent_uuid: "fa56c467fcda49a98d8057fa2c53cc4b" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43389 } } peers { permanent_uuid: "e1a6fe3acd834ed5a893cffea0b7dc22" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 34683 } }
I20260927 18:39:14.416457 16851 raft_consensus.cc:374] T b43f63c9f4c34c55834cd5092ed4ea6f P e1a6fe3acd834ed5a893cffea0b7dc22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:14.416484 16851 raft_consensus.cc:729] T b43f63c9f4c34c55834cd5092ed4ea6f P e1a6fe3acd834ed5a893cffea0b7dc22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1a6fe3acd834ed5a893cffea0b7dc22, State: Initialized, Role: FOLLOWER
I20260927 18:39:14.416559 16851 consensus_queue.cc:260] T b43f63c9f4c34c55834cd5092ed4ea6f P e1a6fe3acd834ed5a893cffea0b7dc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2cdf0d8e2884a83b6a642a3b25681e8" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41855 } } peers { permanent_uuid: "fa56c467fcda49a98d8057fa2c53cc4b" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43389 } } peers { permanent_uuid: "e1a6fe3acd834ed5a893cffea0b7dc22" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 34683 } }
I20260927 18:39:14.416760 16851 ts_tablet_manager.cc:1428] T b43f63c9f4c34c55834cd5092ed4ea6f P e1a6fe3acd834ed5a893cffea0b7dc22: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:39:14.416834 16831 heartbeater.cc:503] Master 127.11.251.254:41063 was elected leader, sending a full tablet report...
W20260927 18:39:14.440827 16699 tablet.cc:2369] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:39:14.511628 16565 tablet.cc:2369] T b43f63c9f4c34c55834cd5092ed4ea6f P fa56c467fcda49a98d8057fa2c53cc4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:39:14.563185 16855 raft_consensus.cc:482] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:39:14.563349 16855 raft_consensus.cc:504] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2cdf0d8e2884a83b6a642a3b25681e8" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41855 } } peers { permanent_uuid: "fa56c467fcda49a98d8057fa2c53cc4b" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43389 } } peers { permanent_uuid: "e1a6fe3acd834ed5a893cffea0b7dc22" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 34683 } }
I20260927 18:39:14.563822 16855 leader_election.cc:288] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa56c467fcda49a98d8057fa2c53cc4b (127.11.251.193:43389), e1a6fe3acd834ed5a893cffea0b7dc22 (127.11.251.195:34683)
I20260927 18:39:14.568667 16845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.564350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:41823 (local address 127.11.251.195:34683)
0927 18:39:14.564437 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:14.564439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:14.564602 (+ 163us) server_negotiation.cc:407] Connection header received
0927 18:39:14.564647 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:14.564648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:14.564662 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:14.564741 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:14.564914 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.565775 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.567528 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.568189 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.568272 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:14.568285 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:14.568515 (+ 230us) server_negotiation.cc:299] Negotiation successful
0927 18:39:14.568567 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 18:39:14.568667 16846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.564336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:39541 (local address 127.11.251.193:43389)
0927 18:39:14.564456 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:14.564461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:14.564487 (+ 26us) server_negotiation.cc:407] Connection header received
0927 18:39:14.564525 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:14.564526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:14.564537 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:14.564639 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:14.564914 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.565774 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.567528 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.568162 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.568272 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:14.568285 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:14.568516 (+ 231us) server_negotiation.cc:299] Negotiation successful
0927 18:39:14.568567 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:39:14.568655 16700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.564141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:43389 (local address 127.11.251.194:39541)
0927 18:39:14.564261 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:14.564276 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:14.564367 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:14.564669 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:14.564669 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:14.564671 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:14.564773 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:14.564776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.565910 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.565911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:14.567384 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:14.567389 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.568308 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.568309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:14.568406 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:14.568409 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:39:14.568484 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 18:39:14.568543 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":119}
I20260927 18:39:14.568660 16857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.564141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:34683 (local address 127.11.251.194:41823)
0927 18:39:14.564510 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:14.564519 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:14.564571 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:14.564763 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:14.564763 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:14.564765 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:14.564821 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:14.564822 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.565910 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.565911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:14.567384 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:14.567389 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.568302 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.568303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:14.568403 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:14.568409 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:39:14.568484 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 18:39:14.568537 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":101,"thread_start_us":204,"threads_started":1}
I20260927 18:39:14.569192 16517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b43f63c9f4c34c55834cd5092ed4ea6f" candidate_uuid: "e2cdf0d8e2884a83b6a642a3b25681e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa56c467fcda49a98d8057fa2c53cc4b" is_pre_election: true
I20260927 18:39:14.569469 16517 raft_consensus.cc:2454] T b43f63c9f4c34c55834cd5092ed4ea6f P fa56c467fcda49a98d8057fa2c53cc4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2cdf0d8e2884a83b6a642a3b25681e8 in term 0.
I20260927 18:39:14.569536 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b43f63c9f4c34c55834cd5092ed4ea6f" candidate_uuid: "e2cdf0d8e2884a83b6a642a3b25681e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1a6fe3acd834ed5a893cffea0b7dc22" is_pre_election: true
I20260927 18:39:14.569784 16785 raft_consensus.cc:2454] T b43f63c9f4c34c55834cd5092ed4ea6f P e1a6fe3acd834ed5a893cffea0b7dc22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2cdf0d8e2884a83b6a642a3b25681e8 in term 0.
I20260927 18:39:14.569800 16588 leader_election.cc:302] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e2cdf0d8e2884a83b6a642a3b25681e8, fa56c467fcda49a98d8057fa2c53cc4b; no voters:
I20260927 18:39:14.569931 16855 raft_consensus.cc:2781] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:39:14.570011 16855 raft_consensus.cc:482] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:39:14.570041 16855 raft_consensus.cc:3037] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:14.570843 16855 raft_consensus.cc:504] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2cdf0d8e2884a83b6a642a3b25681e8" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41855 } } peers { permanent_uuid: "fa56c467fcda49a98d8057fa2c53cc4b" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43389 } } peers { permanent_uuid: "e1a6fe3acd834ed5a893cffea0b7dc22" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 34683 } }
I20260927 18:39:14.571017 16855 leader_election.cc:288] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 [CANDIDATE]: Term 1 election: Requested vote from peers fa56c467fcda49a98d8057fa2c53cc4b (127.11.251.193:43389), e1a6fe3acd834ed5a893cffea0b7dc22 (127.11.251.195:34683)
I20260927 18:39:14.571169 16517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b43f63c9f4c34c55834cd5092ed4ea6f" candidate_uuid: "e2cdf0d8e2884a83b6a642a3b25681e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa56c467fcda49a98d8057fa2c53cc4b"
I20260927 18:39:14.571267 16517 raft_consensus.cc:3037] T b43f63c9f4c34c55834cd5092ed4ea6f P fa56c467fcda49a98d8057fa2c53cc4b [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:14.571271 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b43f63c9f4c34c55834cd5092ed4ea6f" candidate_uuid: "e2cdf0d8e2884a83b6a642a3b25681e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1a6fe3acd834ed5a893cffea0b7dc22"
I20260927 18:39:14.571388 16785 raft_consensus.cc:3037] T b43f63c9f4c34c55834cd5092ed4ea6f P e1a6fe3acd834ed5a893cffea0b7dc22 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:14.572232 16517 raft_consensus.cc:2454] T b43f63c9f4c34c55834cd5092ed4ea6f P fa56c467fcda49a98d8057fa2c53cc4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2cdf0d8e2884a83b6a642a3b25681e8 in term 1.
I20260927 18:39:14.572397 16588 leader_election.cc:302] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e2cdf0d8e2884a83b6a642a3b25681e8, fa56c467fcda49a98d8057fa2c53cc4b; no voters:
I20260927 18:39:14.572427 16785 raft_consensus.cc:2454] T b43f63c9f4c34c55834cd5092ed4ea6f P e1a6fe3acd834ed5a893cffea0b7dc22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2cdf0d8e2884a83b6a642a3b25681e8 in term 1.
I20260927 18:39:14.572482 16855 raft_consensus.cc:2781] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:14.572647 16855 raft_consensus.cc:686] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 [term 1 LEADER]: Becoming Leader. State: Replica: e2cdf0d8e2884a83b6a642a3b25681e8, State: Running, Role: LEADER
I20260927 18:39:14.572760 16855 consensus_queue.cc:237] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2cdf0d8e2884a83b6a642a3b25681e8" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41855 } } peers { permanent_uuid: "fa56c467fcda49a98d8057fa2c53cc4b" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43389 } } peers { permanent_uuid: "e1a6fe3acd834ed5a893cffea0b7dc22" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 34683 } }
I20260927 18:39:14.573945 16374 catalog_manager.cc:5495] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2cdf0d8e2884a83b6a642a3b25681e8 (127.11.251.194). New cstate: current_term: 1 leader_uuid: "e2cdf0d8e2884a83b6a642a3b25681e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2cdf0d8e2884a83b6a642a3b25681e8" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa56c467fcda49a98d8057fa2c53cc4b" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1a6fe3acd834ed5a893cffea0b7dc22" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 34683 } health_report { overall_health: UNKNOWN } } }
I20260927 18:39:14.588236 16847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.585440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39290 (local address 127.11.251.194:41855)
0927 18:39:14.585549 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:14.585552 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:14.585561 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:39:14.585668 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:14.585669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:14.585686 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:14.585798 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:39:14.586145 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.586627 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.587436 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:14.587627 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:14.587711 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:14.587900 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:39:14.588009 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:39:14.588105 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:14.588125 (+ 20us) server_negotiation.cc:299] Negotiation successful
0927 18:39:14.588146 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260927 18:39:14.590926 16517 raft_consensus.cc:1261] T b43f63c9f4c34c55834cd5092ed4ea6f P fa56c467fcda49a98d8057fa2c53cc4b [term 1 FOLLOWER]: Refusing update from remote peer e2cdf0d8e2884a83b6a642a3b25681e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:14.590931 16785 raft_consensus.cc:1261] T b43f63c9f4c34c55834cd5092ed4ea6f P e1a6fe3acd834ed5a893cffea0b7dc22 [term 1 FOLLOWER]: Refusing update from remote peer e2cdf0d8e2884a83b6a642a3b25681e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:14.591388 16855 consensus_queue.cc:1035] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1a6fe3acd834ed5a893cffea0b7dc22" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 34683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:14.591703 16858 consensus_queue.cc:1035] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa56c467fcda49a98d8057fa2c53cc4b" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:14.597219 16374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:14.597332 16374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:14.598242 16374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91977009379546bca6eb5534bbd2a214: Sending DeleteTablet for 3 replicas of tablet b43f63c9f4c34c55834cd5092ed4ea6f
I20260927 18:39:14.598641 16632 tablet_service.cc:1514] Processing DeleteTablet for tablet b43f63c9f4c34c55834cd5092ed4ea6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:14 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:39278
I20260927 18:39:14.598657 16498 tablet_service.cc:1514] Processing DeleteTablet for tablet b43f63c9f4c34c55834cd5092ed4ea6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:14 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:57274
I20260927 18:39:14.598692 16765 tablet_service.cc:1514] Processing DeleteTablet for tablet b43f63c9f4c34c55834cd5092ed4ea6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:14 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:59456
I20260927 18:39:14.598923 16872 tablet_replica.cc:330] T b43f63c9f4c34c55834cd5092ed4ea6f P fa56c467fcda49a98d8057fa2c53cc4b: stopping tablet replica
I20260927 18:39:14.599071 16872 raft_consensus.cc:2229] T b43f63c9f4c34c55834cd5092ed4ea6f P fa56c467fcda49a98d8057fa2c53cc4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:14.599071 16871 tablet_replica.cc:330] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8: stopping tablet replica
I20260927 18:39:14.599166 16871 raft_consensus.cc:2229] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:39:14.599275 16872 raft_consensus.cc:2258] T b43f63c9f4c34c55834cd5092ed4ea6f P fa56c467fcda49a98d8057fa2c53cc4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:14.599309 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 16436
I20260927 18:39:14.599334 16871 raft_consensus.cc:2258] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:14.601734 16871 ts_tablet_manager.cc:1905] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:14.602981 16873 tablet_replica.cc:330] T b43f63c9f4c34c55834cd5092ed4ea6f P e1a6fe3acd834ed5a893cffea0b7dc22: stopping tablet replica
I20260927 18:39:14.603156 16873 raft_consensus.cc:2229] T b43f63c9f4c34c55834cd5092ed4ea6f P e1a6fe3acd834ed5a893cffea0b7dc22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:14.603328 16873 raft_consensus.cc:2258] T b43f63c9f4c34c55834cd5092ed4ea6f P e1a6fe3acd834ed5a893cffea0b7dc22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:14.603677 16871 ts_tablet_manager.cc:1918] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:39:14.603829 16871 log.cc:1196] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/b43f63c9f4c34c55834cd5092ed4ea6f
I20260927 18:39:14.603917 16873 ts_tablet_manager.cc:1905] T b43f63c9f4c34c55834cd5092ed4ea6f P e1a6fe3acd834ed5a893cffea0b7dc22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:14.604240 16871 ts_tablet_manager.cc:1939] T b43f63c9f4c34c55834cd5092ed4ea6f P e2cdf0d8e2884a83b6a642a3b25681e8: Deleting consensus metadata
I20260927 18:39:14.604748 16358 catalog_manager.cc:4841] TS e2cdf0d8e2884a83b6a642a3b25681e8 (127.11.251.194:41855): tablet b43f63c9f4c34c55834cd5092ed4ea6f (table test-table [id=8a60fffd054f48cebf987704fc6f5d5e]) successfully deleted
W20260927 18:39:14.605620 16361 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.251.193:43389 (error 108)
I20260927 18:39:14.606207 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 16570
I20260927 18:39:14.606221 16843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.606083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:43389 (local address 127.0.0.1:57286)
0927 18:39:14.606135 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:14.606183 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.193:43389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260927 18:39:14.607714 16873 ts_tablet_manager.cc:1918] T b43f63c9f4c34c55834cd5092ed4ea6f P e1a6fe3acd834ed5a893cffea0b7dc22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:39:14.607820 16873 log.cc:1196] T b43f63c9f4c34c55834cd5092ed4ea6f P e1a6fe3acd834ed5a893cffea0b7dc22: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/b43f63c9f4c34c55834cd5092ed4ea6f
I20260927 18:39:14.608147 16873 ts_tablet_manager.cc:1939] T b43f63c9f4c34c55834cd5092ed4ea6f P e1a6fe3acd834ed5a893cffea0b7dc22: Deleting consensus metadata
W20260927 18:39:14.608592 16361 catalog_manager.cc:4584] TS fa56c467fcda49a98d8057fa2c53cc4b (127.11.251.193:43389): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b43f63c9f4c34c55834cd5092ed4ea6f: Network error: Client connection negotiation failed: client connection to 127.11.251.193:43389: connect: Connection refused (error 111)
I20260927 18:39:14.608624 16359 catalog_manager.cc:4841] TS e1a6fe3acd834ed5a893cffea0b7dc22 (127.11.251.195:34683): tablet b43f63c9f4c34c55834cd5092ed4ea6f (table test-table [id=8a60fffd054f48cebf987704fc6f5d5e]) successfully deleted
I20260927 18:39:14.612780 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 16703
I20260927 18:39:14.618647 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 16343
2026-09-27T18:39:14Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (5472 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-27T18:39:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:39:14Z Disabled control of system clock
I20260927 18:39:14.649827 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:46841
--webserver_interface=127.11.251.254
--webserver_port=0
--builtin_ntp_servers=127.11.251.212:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:46841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 18:39:14.736428 16878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:14.736640 16878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:14.736661 16878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:14.737866 16878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:14.737918 16878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:14.737931 16878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:14.737944 16878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:14.737957 16878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:39:14.739337 16878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:46841
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:46841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:14.739598 16878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:14.739902 16878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:14.744964 16886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:14.745121 16884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:14.744967 16883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:14.745401 16878 server_base.cc:1034] running on GCE node
I20260927 18:39:14.745783 16878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:14.746112 16878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:14.747339 16878 hybrid_clock.cc:648] HybridClock initialized: now 1790534354747310 us; error 52 us; skew 500 ppm
I20260927 18:39:14.748867 16878 webserver.cc:460] Webserver started at http://127.11.251.254:41437/ using document root <none> and password file <none>
I20260927 18:39:14.749152 16878 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:14.749225 16878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:14.749408 16878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:14.750808 16878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "0d8698e28360482ab44be7942c9e904d"
format_stamp: "Formatted at 2026-09-27 18:39:14 on dist-test-slave-z47j"
server_key: "d126eaa70c59e75a009df62843e97d13"
server_key_iv: "eec28fd59e25c8b0da8a5a967210db4d"
server_key_version: "encryptionkey@0"
I20260927 18:39:14.751230 16878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "0d8698e28360482ab44be7942c9e904d"
format_stamp: "Formatted at 2026-09-27 18:39:14 on dist-test-slave-z47j"
server_key: "d126eaa70c59e75a009df62843e97d13"
server_key_iv: "eec28fd59e25c8b0da8a5a967210db4d"
server_key_version: "encryptionkey@0"
I20260927 18:39:14.752823 16878 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 18:39:14.753805 16892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:14.754047 16878 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:14.754163 16878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "0d8698e28360482ab44be7942c9e904d"
format_stamp: "Formatted at 2026-09-27 18:39:14 on dist-test-slave-z47j"
server_key: "d126eaa70c59e75a009df62843e97d13"
server_key_iv: "eec28fd59e25c8b0da8a5a967210db4d"
server_key_version: "encryptionkey@0"
I20260927 18:39:14.754243 16878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:14.757861 16878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:14.758172 16878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:14.758320 16878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:14.763278 16878 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:46841
I20260927 18:39:14.763319 16944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:46841 every 8 connection(s)
I20260927 18:39:14.763686 16878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:39:14.764434 16945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:14.764961 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 16878
I20260927 18:39:14.765064 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:39:14.765300 12271 external_mini_cluster.cc:1428] Setting key fb0cc08d2673cd702ab7dc0269c35739
I20260927 18:39:14.768436 16948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.766009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50996 (local address 127.11.251.254:46841)
0927 18:39:14.766306 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:14.766316 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:14.766352 (+ 36us) server_negotiation.cc:407] Connection header received
0927 18:39:14.766877 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:14.766886 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:14.766943 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:14.767140 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:14.767874 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:14.767878 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:14.767878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:14.767883 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:14.767937 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:14.767951 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:14.768038 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:14.768177 (+ 139us) server_negotiation.cc:299] Negotiation successful
0927 18:39:14.768240 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":128,"threads_started":1}
I20260927 18:39:14.768877 16945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d: Bootstrap starting.
I20260927 18:39:14.769708 16945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:14.769951 16945 log.cc:824] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:14.770745 16945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d: No bootstrap required, opened a new log
I20260927 18:39:14.772099 16945 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8698e28360482ab44be7942c9e904d" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46841 } }
I20260927 18:39:14.772250 16945 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:14.772274 16945 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d8698e28360482ab44be7942c9e904d, State: Initialized, Role: FOLLOWER
I20260927 18:39:14.772401 16945 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8698e28360482ab44be7942c9e904d" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46841 } }
I20260927 18:39:14.772498 16945 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:39:14.772535 16945 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:39:14.772565 16945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:14.773380 16945 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8698e28360482ab44be7942c9e904d" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46841 } }
I20260927 18:39:14.773470 16945 leader_election.cc:302] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d8698e28360482ab44be7942c9e904d; no voters:
I20260927 18:39:14.773696 16945 leader_election.cc:288] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:39:14.773818 16950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:14.774012 16945 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:14.774016 16950 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d [term 1 LEADER]: Becoming Leader. State: Replica: 0d8698e28360482ab44be7942c9e904d, State: Running, Role: LEADER
I20260927 18:39:14.774183 16950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8698e28360482ab44be7942c9e904d" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46841 } }
I20260927 18:39:14.774660 16952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d8698e28360482ab44be7942c9e904d. Latest consensus state: current_term: 1 leader_uuid: "0d8698e28360482ab44be7942c9e904d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8698e28360482ab44be7942c9e904d" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46841 } } }
I20260927 18:39:14.774684 16951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d8698e28360482ab44be7942c9e904d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8698e28360482ab44be7942c9e904d" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46841 } } }
I20260927 18:39:14.774809 16952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:14.774829 16951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:14.775718 16959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:39:14.776391 16959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:39:14.778141 16959 catalog_manager.cc:1347] Generated new cluster ID: 1b14fbe4497c447d97a494a19816c8ab
I20260927 18:39:14.778199 16959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:39:14.793445 16959 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:39:14.793702 16959 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:39:14.797567 16959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d: Generated new TSK 0
I20260927 18:39:14.797794 16959 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:39:14.806687 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:46841
--builtin_ntp_servers=127.11.251.212:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 18:39:14.899694 16969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:14.899951 16969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:14.899977 16969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:14.901193 16969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:14.901237 16969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:14.901265 16969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:39:14.902781 16969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:46841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:14.903029 16969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:14.903323 16969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:14.908982 16977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:14.908959 16975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:14.908965 16974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:14.909082 16969 server_base.cc:1034] running on GCE node
I20260927 18:39:14.909528 16969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:14.909837 16969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:14.911033 16969 hybrid_clock.cc:648] HybridClock initialized: now 1790534354911031 us; error 69 us; skew 500 ppm
I20260927 18:39:14.912499 16969 webserver.cc:460] Webserver started at http://127.11.251.193:37503/ using document root <none> and password file <none>
I20260927 18:39:14.912768 16969 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:14.912839 16969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:14.913002 16969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:14.914206 16969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "55efac2c01b44ba88d96ae3c98f24a5f"
format_stamp: "Formatted at 2026-09-27 18:39:14 on dist-test-slave-z47j"
server_key: "a7364dc89590745f3a4a266c5790b456"
server_key_iv: "5f374a13f07b8e5b311365d42499b5b6"
server_key_version: "encryptionkey@0"
I20260927 18:39:14.914572 16969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "55efac2c01b44ba88d96ae3c98f24a5f"
format_stamp: "Formatted at 2026-09-27 18:39:14 on dist-test-slave-z47j"
server_key: "a7364dc89590745f3a4a266c5790b456"
server_key_iv: "5f374a13f07b8e5b311365d42499b5b6"
server_key_version: "encryptionkey@0"
I20260927 18:39:14.915983 16969 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260927 18:39:14.916888 16983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:14.917191 16969 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:14.917307 16969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "55efac2c01b44ba88d96ae3c98f24a5f"
format_stamp: "Formatted at 2026-09-27 18:39:14 on dist-test-slave-z47j"
server_key: "a7364dc89590745f3a4a266c5790b456"
server_key_iv: "5f374a13f07b8e5b311365d42499b5b6"
server_key_version: "encryptionkey@0"
I20260927 18:39:14.917387 16969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:14.920804 16969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:14.921075 16969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:14.921226 16969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:14.921445 16969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:14.921818 16969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:14.921872 16969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:14.921906 16969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:14.921933 16969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:14.928591 16969 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:46861
I20260927 18:39:14.928655 17096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:46861 every 8 connection(s)
I20260927 18:39:14.928977 16969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:39:14.932545 16948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.929945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:56437 (local address 127.11.251.254:46841)
0927 18:39:14.930025 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:14.930028 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:14.930377 (+ 349us) server_negotiation.cc:407] Connection header received
0927 18:39:14.930941 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:14.930943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:14.930948 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:14.931033 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:14.932311 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:14.932312 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:14.932313 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:14.932317 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:14.932336 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:14.932340 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:14.932403 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:14.932477 (+ 74us) server_negotiation.cc:299] Negotiation successful
0927 18:39:14.932491 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 18:39:14.932807 17099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:14.929924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:46841 (local address 127.11.251.193:56437)
0927 18:39:14.930170 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:14.930218 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:14.930801 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:14.931117 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:14.931121 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:14.931131 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:14.932140 (+ 1009us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:39:14.932141 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:14.932174 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:39:14.932207 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:14.932402 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:14.932405 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:14.932408 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:39:14.932468 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 18:39:14.932527 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":123,"threads_started":1}
I20260927 18:39:14.933432 17097 heartbeater.cc:347] Connected to a master server at 127.11.251.254:46841
I20260927 18:39:14.933477 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 16969
I20260927 18:39:14.933557 17097 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:14.933643 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:39:14.933758 17097 heartbeater.cc:511] Master 127.11.251.254:46841 requested a full tablet report, sending...
I20260927 18:39:14.933883 12271 external_mini_cluster.cc:1428] Setting key 8d1c67e2bfba5e7510600c467dba9e7c
I20260927 18:39:14.934499 16909 ts_manager.cc:194] Registered new tserver with Master: 55efac2c01b44ba88d96ae3c98f24a5f (127.11.251.193:46861)
I20260927 18:39:14.935153 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:46841
--builtin_ntp_servers=127.11.251.212:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 18:39:15.021425 17100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:15.021703 17100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:15.021728 17100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:15.022872 17100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:15.022913 17100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:15.022938 17100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:39:15.024190 17100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:46841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:15.024390 17100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:15.024623 17100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:15.030359 17106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:15.030463 17100 server_base.cc:1034] running on GCE node
W20260927 18:39:15.030572 17105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:15.030438 17108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:15.030838 17100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:15.031147 17100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:15.032362 17100 hybrid_clock.cc:648] HybridClock initialized: now 1790534355032331 us; error 66 us; skew 500 ppm
I20260927 18:39:15.034202 17100 webserver.cc:460] Webserver started at http://127.11.251.194:36561/ using document root <none> and password file <none>
I20260927 18:39:15.034513 17100 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:15.034598 17100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:15.034780 17100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:15.036118 17100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "cd226c8804874240a95d3cc065d381b3"
format_stamp: "Formatted at 2026-09-27 18:39:15 on dist-test-slave-z47j"
server_key: "17c2c89cb196e3aa30c7d1f685b095cd"
server_key_iv: "7861c9d49bc505177c7b4d674346a4c2"
server_key_version: "encryptionkey@0"
I20260927 18:39:15.036550 17100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "cd226c8804874240a95d3cc065d381b3"
format_stamp: "Formatted at 2026-09-27 18:39:15 on dist-test-slave-z47j"
server_key: "17c2c89cb196e3aa30c7d1f685b095cd"
server_key_iv: "7861c9d49bc505177c7b4d674346a4c2"
server_key_version: "encryptionkey@0"
I20260927 18:39:15.038254 17100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260927 18:39:15.039179 17114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:15.039469 17100 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:15.039592 17100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "cd226c8804874240a95d3cc065d381b3"
format_stamp: "Formatted at 2026-09-27 18:39:15 on dist-test-slave-z47j"
server_key: "17c2c89cb196e3aa30c7d1f685b095cd"
server_key_iv: "7861c9d49bc505177c7b4d674346a4c2"
server_key_version: "encryptionkey@0"
I20260927 18:39:15.039687 17100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:15.043474 17100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:15.043776 17100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:15.043922 17100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:15.044158 17100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:15.044489 17100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:15.044545 17100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:15.044588 17100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:15.044615 17100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:15.052237 17100 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:37093
I20260927 18:39:15.052296 17227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:37093 every 8 connection(s)
I20260927 18:39:15.052711 17100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:39:15.056437 16948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:15.053631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:38311 (local address 127.11.251.254:46841)
0927 18:39:15.053717 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:15.053721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:15.053942 (+ 221us) server_negotiation.cc:407] Connection header received
0927 18:39:15.054579 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:15.054581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:15.054589 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:15.054653 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:15.056081 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:15.056083 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:15.056084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:15.056090 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:15.056109 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:15.056114 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:15.056206 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:15.056325 (+ 119us) server_negotiation.cc:299] Negotiation successful
0927 18:39:15.056344 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:39:15.056720 17230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:15.053493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:46841 (local address 127.11.251.194:38311)
0927 18:39:15.053782 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:15.053822 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:15.054425 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:15.054745 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:15.054749 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:15.054759 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:15.055883 (+ 1124us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:39:15.055883 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:15.055918 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:39:15.055950 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:15.056205 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:15.056213 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:15.056216 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:39:15.056336 (+ 120us) client_negotiation.cc:240] Negotiation successful
0927 18:39:15.056445 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":156,"threads_started":1}
I20260927 18:39:15.057420 17228 heartbeater.cc:347] Connected to a master server at 127.11.251.254:46841
I20260927 18:39:15.057538 17228 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:15.057762 17228 heartbeater.cc:511] Master 127.11.251.254:46841 requested a full tablet report, sending...
I20260927 18:39:15.058372 16909 ts_manager.cc:194] Registered new tserver with Master: cd226c8804874240a95d3cc065d381b3 (127.11.251.194:37093)
I20260927 18:39:15.060685 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 17100
I20260927 18:39:15.060776 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:39:15.060918 12271 external_mini_cluster.cc:1428] Setting key 3de8e2b69bbcc9801aedfbdcaf9abfe7
I20260927 18:39:15.062053 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:46841
--builtin_ntp_servers=127.11.251.212:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 18:39:15.163968 17231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:15.164345 17231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:15.164388 17231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:15.166245 17231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:15.166293 17231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:15.166318 17231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:39:15.167564 17231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:46841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:15.167745 17231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:15.167968 17231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:15.173779 17237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:15.173765 17236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:15.173763 17239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:15.174254 17231 server_base.cc:1034] running on GCE node
I20260927 18:39:15.174544 17231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:15.174850 17231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:15.176028 17231 hybrid_clock.cc:648] HybridClock initialized: now 1790534355176010 us; error 42 us; skew 500 ppm
I20260927 18:39:15.177547 17231 webserver.cc:460] Webserver started at http://127.11.251.195:36657/ using document root <none> and password file <none>
I20260927 18:39:15.177872 17231 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:15.177922 17231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:15.178125 17231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:15.179350 17231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "db530c485da3423d94358d80f1345bca"
format_stamp: "Formatted at 2026-09-27 18:39:15 on dist-test-slave-z47j"
server_key: "a4d434de8bc3b0befde14395b012243c"
server_key_iv: "d8b1ae9c95dd02e4c60a67b669eb941c"
server_key_version: "encryptionkey@0"
I20260927 18:39:15.179742 17231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "db530c485da3423d94358d80f1345bca"
format_stamp: "Formatted at 2026-09-27 18:39:15 on dist-test-slave-z47j"
server_key: "a4d434de8bc3b0befde14395b012243c"
server_key_iv: "d8b1ae9c95dd02e4c60a67b669eb941c"
server_key_version: "encryptionkey@0"
I20260927 18:39:15.181206 17231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:39:15.182193 17245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:15.182466 17231 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 18:39:15.182574 17231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "db530c485da3423d94358d80f1345bca"
format_stamp: "Formatted at 2026-09-27 18:39:15 on dist-test-slave-z47j"
server_key: "a4d434de8bc3b0befde14395b012243c"
server_key_iv: "d8b1ae9c95dd02e4c60a67b669eb941c"
server_key_version: "encryptionkey@0"
I20260927 18:39:15.182663 17231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:15.186363 17231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:15.186650 17231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:15.186791 17231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:15.187037 17231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:15.187357 17231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:15.187412 17231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:15.187443 17231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:15.187471 17231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:15.194645 17231 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:35761
I20260927 18:39:15.194692 17358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:35761 every 8 connection(s)
I20260927 18:39:15.195034 17231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:39:15.197369 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 17231
I20260927 18:39:15.197505 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:39:15.197784 12271 external_mini_cluster.cc:1428] Setting key 8efe1ef4a1e99a94d7cb69bf9a380e16
I20260927 18:39:15.198761 16948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:15.195817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:42003 (local address 127.11.251.254:46841)
0927 18:39:15.195915 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:15.195918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:15.196178 (+ 260us) server_negotiation.cc:407] Connection header received
0927 18:39:15.196761 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:15.196762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:15.196768 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:15.196836 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:15.198411 (+ 1575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:15.198412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:15.198412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:15.198417 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:15.198436 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:15.198440 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:15.198508 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:15.198629 (+ 121us) server_negotiation.cc:299] Negotiation successful
0927 18:39:15.198647 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 18:39:15.198982 17361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:15.195780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:46841 (local address 127.11.251.195:42003)
0927 18:39:15.196017 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:15.196062 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:15.196631 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:15.196918 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:15.196922 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:15.196931 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:15.198207 (+ 1276us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:39:15.198208 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:15.198247 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:39:15.198305 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:15.198528 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:15.198532 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:15.198536 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:39:15.198624 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 18:39:15.198694 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":124,"threads_started":1}
I20260927 18:39:15.199446 17359 heartbeater.cc:347] Connected to a master server at 127.11.251.254:46841
I20260927 18:39:15.199529 17359 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:15.199714 17359 heartbeater.cc:511] Master 127.11.251.254:46841 requested a full tablet report, sending...
I20260927 18:39:15.200085 16909 ts_manager.cc:194] Registered new tserver with Master: db530c485da3423d94358d80f1345bca (127.11.251.195:35761)
I20260927 18:39:15.210249 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:39:15.213013 16948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:15.211751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51010 (local address 127.11.251.254:46841)
0927 18:39:15.211814 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:15.211816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:15.211991 (+ 175us) server_negotiation.cc:407] Connection header received
0927 18:39:15.212066 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:15.212067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:15.212072 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:15.212124 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:15.212804 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:15.212805 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:15.212806 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:15.212810 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:15.212822 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:15.212826 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:15.212870 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:15.212925 (+ 55us) server_negotiation.cc:299] Negotiation successful
0927 18:39:15.212941 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 18:39:15.214653 16909 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51010:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:39:15.216150 16909 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:39:15.220575 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:15.219106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:35761 (local address 127.0.0.1:36202)
0927 18:39:15.219221 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:15.219230 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:15.219267 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:15.219752 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:15.219753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:15.219757 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:15.220167 (+ 410us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:39:15.220168 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:15.220183 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:39:15.220200 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:15.220436 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:15.220442 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:15.220446 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:39:15.220494 (+ 48us) client_negotiation.cc:240] Negotiation successful
0927 18:39:15.220510 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":73,"threads_started":1}
I20260927 18:39:15.220798 17367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:15.218679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:37093 (local address 127.0.0.1:58226)
0927 18:39:15.218839 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:15.218884 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:15.218959 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:15.219889 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:15.219890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:15.219894 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:15.220347 (+ 453us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:39:15.220348 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:15.220358 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:39:15.220372 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:15.220640 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:15.220641 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:15.220642 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 18:39:15.220690 (+ 48us) client_negotiation.cc:240] Negotiation successful
0927 18:39:15.220705 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
I20260927 18:39:15.220845 17372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:15.219348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36202 (local address 127.11.251.195:35761)
0927 18:39:15.219572 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:15.219581 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:15.219589 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:39:15.219609 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:15.219611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:15.219618 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:15.219745 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:15.220291 (+ 546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:15.220296 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:15.220297 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:15.220302 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:15.220339 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:15.220359 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:15.220729 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:15.220755 (+ 26us) server_negotiation.cc:299] Negotiation successful
0927 18:39:15.220771 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":78,"threads_started":1}
I20260927 18:39:15.220991 17369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:15.218913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:46861 (local address 127.0.0.1:56536)
0927 18:39:15.219058 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:15.219068 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:15.219109 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:15.219531 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:15.219538 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:15.219547 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:15.220546 (+ 999us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:39:15.220546 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:15.220554 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:39:15.220565 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:15.220868 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:15.220869 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:15.220869 (+ 0us) client_negotiation.cc:769] Sending connection context
0927 18:39:15.220904 (+ 35us) client_negotiation.cc:240] Negotiation successful
0927 18:39:15.220919 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260927 18:39:15.221144 17368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:15.218851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58226 (local address 127.11.251.194:37093)
0927 18:39:15.219139 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:15.219144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:15.219159 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:39:15.219358 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:15.219363 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:15.219376 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:15.219456 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:15.220465 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:15.220471 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:15.220472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:15.220476 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:15.220519 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:15.220554 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:15.220616 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:15.221025 (+ 409us) server_negotiation.cc:299] Negotiation successful
0927 18:39:15.221043 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":137,"threads_started":1}
I20260927 18:39:15.221251 17370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:15.219035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56536 (local address 127.11.251.193:46861)
0927 18:39:15.219361 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:15.219364 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:15.219375 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:39:15.219406 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:15.219410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:15.219418 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:15.220337 (+ 919us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:15.220744 (+ 407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:15.220747 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:15.220747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:15.220750 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:15.220779 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:15.220790 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:15.221102 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:15.221125 (+ 23us) server_negotiation.cc:299] Negotiation successful
0927 18:39:15.221145 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":116,"threads_started":1}
I20260927 18:39:15.221925 17293 tablet_service.cc:1467] Processing CreateTablet for tablet a968a15ca713489f9e4bf3820f1f0d5a (DEFAULT_TABLE table=test-table [id=205c1296c1314f179a1a7a0e5fc61faa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:15.222028 17031 tablet_service.cc:1467] Processing CreateTablet for tablet a968a15ca713489f9e4bf3820f1f0d5a (DEFAULT_TABLE table=test-table [id=205c1296c1314f179a1a7a0e5fc61faa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:15.222025 17162 tablet_service.cc:1467] Processing CreateTablet for tablet a968a15ca713489f9e4bf3820f1f0d5a (DEFAULT_TABLE table=test-table [id=205c1296c1314f179a1a7a0e5fc61faa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:15.222322 17293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a968a15ca713489f9e4bf3820f1f0d5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:15.222442 17031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a968a15ca713489f9e4bf3820f1f0d5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:15.222442 17162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a968a15ca713489f9e4bf3820f1f0d5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:15.224937 17373 tablet_bootstrap.cc:492] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca: Bootstrap starting.
I20260927 18:39:15.225548 17373 tablet_bootstrap.cc:654] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:15.225908 17373 log.cc:824] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:15.226188 17375 tablet_bootstrap.cc:492] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3: Bootstrap starting.
I20260927 18:39:15.226481 17374 tablet_bootstrap.cc:492] T a968a15ca713489f9e4bf3820f1f0d5a P 55efac2c01b44ba88d96ae3c98f24a5f: Bootstrap starting.
I20260927 18:39:15.226647 17373 tablet_bootstrap.cc:492] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca: No bootstrap required, opened a new log
I20260927 18:39:15.226730 17373 ts_tablet_manager.cc:1397] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 18:39:15.227197 17374 tablet_bootstrap.cc:654] T a968a15ca713489f9e4bf3820f1f0d5a P 55efac2c01b44ba88d96ae3c98f24a5f: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:15.227532 17374 log.cc:824] T a968a15ca713489f9e4bf3820f1f0d5a P 55efac2c01b44ba88d96ae3c98f24a5f: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:15.228310 17374 tablet_bootstrap.cc:492] T a968a15ca713489f9e4bf3820f1f0d5a P 55efac2c01b44ba88d96ae3c98f24a5f: No bootstrap required, opened a new log
I20260927 18:39:15.228397 17375 tablet_bootstrap.cc:654] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:15.228410 17374 ts_tablet_manager.cc:1397] T a968a15ca713489f9e4bf3820f1f0d5a P 55efac2c01b44ba88d96ae3c98f24a5f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:39:15.228734 17375 log.cc:824] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:15.229645 17375 tablet_bootstrap.cc:492] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3: No bootstrap required, opened a new log
I20260927 18:39:15.229508 17373 raft_consensus.cc:348] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd226c8804874240a95d3cc065d381b3" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37093 } } peers { permanent_uuid: "55efac2c01b44ba88d96ae3c98f24a5f" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46861 } } peers { permanent_uuid: "db530c485da3423d94358d80f1345bca" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35761 } }
I20260927 18:39:15.229693 17373 raft_consensus.cc:374] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:15.229759 17373 raft_consensus.cc:729] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db530c485da3423d94358d80f1345bca, State: Initialized, Role: FOLLOWER
I20260927 18:39:15.229884 17375 ts_tablet_manager.cc:1397] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260927 18:39:15.229878 17373 consensus_queue.cc:260] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd226c8804874240a95d3cc065d381b3" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37093 } } peers { permanent_uuid: "55efac2c01b44ba88d96ae3c98f24a5f" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46861 } } peers { permanent_uuid: "db530c485da3423d94358d80f1345bca" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35761 } }
I20260927 18:39:15.230136 17373 ts_tablet_manager.cc:1428] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260927 18:39:15.230269 17359 heartbeater.cc:503] Master 127.11.251.254:46841 was elected leader, sending a full tablet report...
I20260927 18:39:15.231289 17374 raft_consensus.cc:348] T a968a15ca713489f9e4bf3820f1f0d5a P 55efac2c01b44ba88d96ae3c98f24a5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd226c8804874240a95d3cc065d381b3" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37093 } } peers { permanent_uuid: "55efac2c01b44ba88d96ae3c98f24a5f" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46861 } } peers { permanent_uuid: "db530c485da3423d94358d80f1345bca" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35761 } }
I20260927 18:39:15.231482 17374 raft_consensus.cc:374] T a968a15ca713489f9e4bf3820f1f0d5a P 55efac2c01b44ba88d96ae3c98f24a5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:15.231520 17374 raft_consensus.cc:729] T a968a15ca713489f9e4bf3820f1f0d5a P 55efac2c01b44ba88d96ae3c98f24a5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55efac2c01b44ba88d96ae3c98f24a5f, State: Initialized, Role: FOLLOWER
I20260927 18:39:15.231649 17374 consensus_queue.cc:260] T a968a15ca713489f9e4bf3820f1f0d5a P 55efac2c01b44ba88d96ae3c98f24a5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd226c8804874240a95d3cc065d381b3" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37093 } } peers { permanent_uuid: "55efac2c01b44ba88d96ae3c98f24a5f" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46861 } } peers { permanent_uuid: "db530c485da3423d94358d80f1345bca" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35761 } }
I20260927 18:39:15.231943 17374 ts_tablet_manager.cc:1428] T a968a15ca713489f9e4bf3820f1f0d5a P 55efac2c01b44ba88d96ae3c98f24a5f: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260927 18:39:15.231969 17097 heartbeater.cc:503] Master 127.11.251.254:46841 was elected leader, sending a full tablet report...
I20260927 18:39:15.232690 17375 raft_consensus.cc:348] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd226c8804874240a95d3cc065d381b3" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37093 } } peers { permanent_uuid: "55efac2c01b44ba88d96ae3c98f24a5f" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46861 } } peers { permanent_uuid: "db530c485da3423d94358d80f1345bca" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35761 } }
I20260927 18:39:15.232812 17375 raft_consensus.cc:374] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:15.232833 17375 raft_consensus.cc:729] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd226c8804874240a95d3cc065d381b3, State: Initialized, Role: FOLLOWER
I20260927 18:39:15.233016 17375 consensus_queue.cc:260] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd226c8804874240a95d3cc065d381b3" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37093 } } peers { permanent_uuid: "55efac2c01b44ba88d96ae3c98f24a5f" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46861 } } peers { permanent_uuid: "db530c485da3423d94358d80f1345bca" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35761 } }
I20260927 18:39:15.233322 17375 ts_tablet_manager.cc:1428] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:39:15.233364 17228 heartbeater.cc:503] Master 127.11.251.254:46841 was elected leader, sending a full tablet report...
W20260927 18:39:15.303315 17229 tablet.cc:2369] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:39:15.429823 17098 tablet.cc:2369] T a968a15ca713489f9e4bf3820f1f0d5a P 55efac2c01b44ba88d96ae3c98f24a5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:39:15.445763 17360 tablet.cc:2369] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:39:15.512132 17381 raft_consensus.cc:482] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:39:15.512269 17381 raft_consensus.cc:504] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd226c8804874240a95d3cc065d381b3" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37093 } } peers { permanent_uuid: "55efac2c01b44ba88d96ae3c98f24a5f" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46861 } } peers { permanent_uuid: "db530c485da3423d94358d80f1345bca" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35761 } }
I20260927 18:39:15.512699 17381 leader_election.cc:288] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55efac2c01b44ba88d96ae3c98f24a5f (127.11.251.193:46861), db530c485da3423d94358d80f1345bca (127.11.251.195:35761)
I20260927 18:39:15.514447 17230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:15.512960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:46861 (local address 127.11.251.194:42727)
0927 18:39:15.513026 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:15.513042 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:15.513090 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:15.513428 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:15.513429 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:15.513432 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:15.514058 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:39:15.514059 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:15.514070 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:39:15.514088 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:15.514316 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:15.514317 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:15.514317 (+ 0us) client_negotiation.cc:769] Sending connection context
0927 18:39:15.514363 (+ 46us) client_negotiation.cc:240] Negotiation successful
0927 18:39:15.514382 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 18:39:15.514601 17370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:15.513048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:42727 (local address 127.11.251.193:46861)
0927 18:39:15.513116 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:15.513118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:15.513136 (+ 18us) server_negotiation.cc:407] Connection header received
0927 18:39:15.513171 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:15.513172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:15.513178 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:15.513297 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:15.514206 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:15.514208 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:15.514208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:15.514213 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:15.514232 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:15.514237 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:15.514308 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:15.514512 (+ 204us) server_negotiation.cc:299] Negotiation successful
0927 18:39:15.514529 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:39:15.514637 17372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:15.513273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:38919 (local address 127.11.251.195:35761)
0927 18:39:15.513357 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:15.513359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:15.513441 (+ 82us) server_negotiation.cc:407] Connection header received
0927 18:39:15.513500 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:15.513501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:15.513507 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:15.513566 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:15.514382 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:15.514383 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:15.514383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:15.514388 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:15.514401 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:15.514406 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:15.514481 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:15.514558 (+ 77us) server_negotiation.cc:299] Negotiation successful
0927 18:39:15.514572 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260927 18:39:15.514820 17382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:15.513092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:35761 (local address 127.11.251.194:38919)
0927 18:39:15.513344 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:15.513358 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:15.513442 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:15.513600 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:15.513601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:15.513604 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:15.514255 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:39:15.514256 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:15.514264 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:39:15.514278 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:15.514478 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:15.514479 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:15.514480 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 18:39:15.514716 (+ 236us) client_negotiation.cc:240] Negotiation successful
0927 18:39:15.514734 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":132,"threads_started":1}
I20260927 18:39:15.515007 17051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a968a15ca713489f9e4bf3820f1f0d5a" candidate_uuid: "cd226c8804874240a95d3cc065d381b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55efac2c01b44ba88d96ae3c98f24a5f" is_pre_election: true
I20260927 18:39:15.515229 17051 raft_consensus.cc:2454] T a968a15ca713489f9e4bf3820f1f0d5a P 55efac2c01b44ba88d96ae3c98f24a5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd226c8804874240a95d3cc065d381b3 in term 0.
I20260927 18:39:15.515532 17115 leader_election.cc:302] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55efac2c01b44ba88d96ae3c98f24a5f, cd226c8804874240a95d3cc065d381b3; no voters:
I20260927 18:39:15.515697 17381 raft_consensus.cc:2781] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:39:15.515765 17381 raft_consensus.cc:482] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:39:15.515803 17381 raft_consensus.cc:3037] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:15.515628 17313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a968a15ca713489f9e4bf3820f1f0d5a" candidate_uuid: "cd226c8804874240a95d3cc065d381b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db530c485da3423d94358d80f1345bca" is_pre_election: true
I20260927 18:39:15.515934 17313 raft_consensus.cc:2454] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd226c8804874240a95d3cc065d381b3 in term 0.
I20260927 18:39:15.516747 17381 raft_consensus.cc:504] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd226c8804874240a95d3cc065d381b3" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37093 } } peers { permanent_uuid: "55efac2c01b44ba88d96ae3c98f24a5f" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46861 } } peers { permanent_uuid: "db530c485da3423d94358d80f1345bca" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35761 } }
I20260927 18:39:15.516911 17381 leader_election.cc:288] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 [CANDIDATE]: Term 1 election: Requested vote from peers 55efac2c01b44ba88d96ae3c98f24a5f (127.11.251.193:46861), db530c485da3423d94358d80f1345bca (127.11.251.195:35761)
I20260927 18:39:15.517118 17313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a968a15ca713489f9e4bf3820f1f0d5a" candidate_uuid: "cd226c8804874240a95d3cc065d381b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db530c485da3423d94358d80f1345bca"
I20260927 18:39:15.517196 17313 raft_consensus.cc:3037] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:15.517148 17051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a968a15ca713489f9e4bf3820f1f0d5a" candidate_uuid: "cd226c8804874240a95d3cc065d381b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55efac2c01b44ba88d96ae3c98f24a5f"
I20260927 18:39:15.517252 17051 raft_consensus.cc:3037] T a968a15ca713489f9e4bf3820f1f0d5a P 55efac2c01b44ba88d96ae3c98f24a5f [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:15.518071 17313 raft_consensus.cc:2454] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd226c8804874240a95d3cc065d381b3 in term 1.
I20260927 18:39:15.518199 17051 raft_consensus.cc:2454] T a968a15ca713489f9e4bf3820f1f0d5a P 55efac2c01b44ba88d96ae3c98f24a5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd226c8804874240a95d3cc065d381b3 in term 1.
I20260927 18:39:15.518239 17115 leader_election.cc:302] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd226c8804874240a95d3cc065d381b3, db530c485da3423d94358d80f1345bca; no voters:
I20260927 18:39:15.518363 17381 raft_consensus.cc:2781] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:15.518558 17381 raft_consensus.cc:686] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 [term 1 LEADER]: Becoming Leader. State: Replica: cd226c8804874240a95d3cc065d381b3, State: Running, Role: LEADER
I20260927 18:39:15.518682 17381 consensus_queue.cc:237] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd226c8804874240a95d3cc065d381b3" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37093 } } peers { permanent_uuid: "55efac2c01b44ba88d96ae3c98f24a5f" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46861 } } peers { permanent_uuid: "db530c485da3423d94358d80f1345bca" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35761 } }
I20260927 18:39:15.519636 16908 catalog_manager.cc:5495] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd226c8804874240a95d3cc065d381b3 (127.11.251.194). New cstate: current_term: 1 leader_uuid: "cd226c8804874240a95d3cc065d381b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd226c8804874240a95d3cc065d381b3" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55efac2c01b44ba88d96ae3c98f24a5f" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db530c485da3423d94358d80f1345bca" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35761 } health_report { overall_health: UNKNOWN } } }
I20260927 18:39:15.579164 17368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:15.577848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58234 (local address 127.11.251.194:37093)
0927 18:39:15.577924 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:15.577926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:15.577935 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:39:15.577984 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:15.577985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:15.578003 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:15.578066 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:15.578922 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:15.578924 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:15.578924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:15.578930 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:15.578945 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:15.578949 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:15.579003 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:15.579070 (+ 67us) server_negotiation.cc:299] Negotiation successful
0927 18:39:15.579082 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:39:15.581385 17051 raft_consensus.cc:1261] T a968a15ca713489f9e4bf3820f1f0d5a P 55efac2c01b44ba88d96ae3c98f24a5f [term 1 FOLLOWER]: Refusing update from remote peer cd226c8804874240a95d3cc065d381b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:15.581451 17313 raft_consensus.cc:1261] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca [term 1 FOLLOWER]: Refusing update from remote peer cd226c8804874240a95d3cc065d381b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:15.581812 17381 consensus_queue.cc:1035] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55efac2c01b44ba88d96ae3c98f24a5f" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:15.581980 17383 consensus_queue.cc:1035] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db530c485da3423d94358d80f1345bca" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:15.585325 17387 mvcc.cc:205] Tried to move back new op lower bound from 7334028720457564160 to 7334028720205029376. Current Snapshot: MvccSnapshot[applied={T|T < 7334028720457564160 or (T in {7334028720457564160})}]
I20260927 18:39:15.585325 17389 mvcc.cc:205] Tried to move back new op lower bound from 7334028720457564160 to 7334028720205029376. Current Snapshot: MvccSnapshot[applied={T|T < 7334028720457564160 or (T in {7334028720457564160})}]
I20260927 18:39:15.587121 16908 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:15.587230 16908 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:15.588246 16908 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d8698e28360482ab44be7942c9e904d: Sending DeleteTablet for 3 replicas of tablet a968a15ca713489f9e4bf3820f1f0d5a
I20260927 18:39:15.588593 17162 tablet_service.cc:1514] Processing DeleteTablet for tablet a968a15ca713489f9e4bf3820f1f0d5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:15 UTC) from {username='slave'} at 127.0.0.1:58226
I20260927 18:39:15.588668 17031 tablet_service.cc:1514] Processing DeleteTablet for tablet a968a15ca713489f9e4bf3820f1f0d5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:15 UTC) from {username='slave'} at 127.0.0.1:56536
I20260927 18:39:15.588689 17293 tablet_service.cc:1514] Processing DeleteTablet for tablet a968a15ca713489f9e4bf3820f1f0d5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:15 UTC) from {username='slave'} at 127.0.0.1:36202
I20260927 18:39:15.588945 17397 tablet_replica.cc:330] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca: stopping tablet replica
I20260927 18:39:15.589044 17397 raft_consensus.cc:2229] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:15.589071 17395 tablet_replica.cc:330] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3: stopping tablet replica
I20260927 18:39:15.589177 17397 raft_consensus.cc:2258] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:15.589179 17395 raft_consensus.cc:2229] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:39:15.589396 17395 raft_consensus.cc:2258] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:15.589406 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 16969
I20260927 18:39:15.589658 17397 ts_tablet_manager.cc:1905] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:15.590732 17395 ts_tablet_manager.cc:1905] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:15.591403 17397 ts_tablet_manager.cc:1918] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:39:15.591478 17397 log.cc:1196] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/a968a15ca713489f9e4bf3820f1f0d5a
I20260927 18:39:15.591754 17397 ts_tablet_manager.cc:1939] T a968a15ca713489f9e4bf3820f1f0d5a P db530c485da3423d94358d80f1345bca: Deleting consensus metadata
I20260927 18:39:15.592170 16893 catalog_manager.cc:4841] TS db530c485da3423d94358d80f1345bca (127.11.251.195:35761): tablet a968a15ca713489f9e4bf3820f1f0d5a (table test-table [id=205c1296c1314f179a1a7a0e5fc61faa]) successfully deleted
I20260927 18:39:15.592772 17395 ts_tablet_manager.cc:1918] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:39:15.592859 17395 log.cc:1196] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/a968a15ca713489f9e4bf3820f1f0d5a
I20260927 18:39:15.593082 17395 ts_tablet_manager.cc:1939] T a968a15ca713489f9e4bf3820f1f0d5a P cd226c8804874240a95d3cc065d381b3: Deleting consensus metadata
I20260927 18:39:15.593458 16893 catalog_manager.cc:4841] TS cd226c8804874240a95d3cc065d381b3 (127.11.251.194:37093): tablet a968a15ca713489f9e4bf3820f1f0d5a (table test-table [id=205c1296c1314f179a1a7a0e5fc61faa]) successfully deleted
W20260927 18:39:15.597815 16893 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.251.193:46861 (error 108)
I20260927 18:39:15.598143 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 17100
I20260927 18:39:15.598296 17369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:15.598173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:46861 (local address 127.0.0.1:56540)
0927 18:39:15.598223 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:15.598259 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.193:46861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260927 18:39:15.600126 16893 catalog_manager.cc:4584] TS 55efac2c01b44ba88d96ae3c98f24a5f (127.11.251.193:46861): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a968a15ca713489f9e4bf3820f1f0d5a: Network error: Client connection negotiation failed: client connection to 127.11.251.193:46861: connect: Connection refused (error 111)
I20260927 18:39:15.605839 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 17231
I20260927 18:39:15.612067 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 16878
2026-09-27T18:39:15Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (992 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260927 18:39:15.633005 12271 mini_oidc.cc:152] Starting JWKS server
I20260927 18:39:15.634356 12271 webserver.cc:460] Webserver started at https://127.0.0.1:43637/ using document root <none> and password file <none>
I20260927 18:39:15.634449 12271 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 18:39:15.634608 12271 webserver.cc:460] Webserver started at http://127.0.0.1:38503/ using document root <none> and password file <none>
2026-09-27T18:39:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:39:15Z Disabled control of system clock
I20260927 18:39:15.644153 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:35547
--webserver_interface=127.11.251.254
--webserver_port=0
--builtin_ntp_servers=127.11.251.212:35681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:35547
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43637/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534329098323-12271-0/testchainca.cert with env {}
W20260927 18:39:15.732069 17404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:15.732326 17404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:15.732352 17404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:15.733645 17404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:15.733695 17404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:15.733706 17404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:15.733721 17404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:15.733732 17404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 18:39:15.733745 17404 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 18:39:15.733757 17404 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43637/jwks.json
I20260927 18:39:15.735442 17404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:35681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:35547
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:35547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43637/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534329098323-12271-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:15.735685 17404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:15.735889 17404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:39:15.741612 17404 server_base.cc:1034] running on GCE node
W20260927 18:39:15.741648 17409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:15.741658 17410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:15.741767 17412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:15.742086 17404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:15.742411 17404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:15.743567 17404 hybrid_clock.cc:648] HybridClock initialized: now 1790534355743543 us; error 40 us; skew 500 ppm
I20260927 18:39:15.744944 17404 webserver.cc:460] Webserver started at http://127.11.251.254:39857/ using document root <none> and password file <none>
I20260927 18:39:15.745251 17404 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:15.745306 17404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:15.745453 17404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:15.746683 17404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "ac6cc9bab4fe4ec1955e4f7f6374ef89"
format_stamp: "Formatted at 2026-09-27 18:39:15 on dist-test-slave-z47j"
server_key: "ac0813fa59fe0919f86ab7fe400de788"
server_key_iv: "fcdfbdff08bd61471e981600a98a4df5"
server_key_version: "encryptionkey@0"
I20260927 18:39:15.747092 17404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "ac6cc9bab4fe4ec1955e4f7f6374ef89"
format_stamp: "Formatted at 2026-09-27 18:39:15 on dist-test-slave-z47j"
server_key: "ac0813fa59fe0919f86ab7fe400de788"
server_key_iv: "fcdfbdff08bd61471e981600a98a4df5"
server_key_version: "encryptionkey@0"
I20260927 18:39:15.748515 17404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:39:15.749377 17418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:15.749694 17404 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:15.749799 17404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "ac6cc9bab4fe4ec1955e4f7f6374ef89"
format_stamp: "Formatted at 2026-09-27 18:39:15 on dist-test-slave-z47j"
server_key: "ac0813fa59fe0919f86ab7fe400de788"
server_key_iv: "fcdfbdff08bd61471e981600a98a4df5"
server_key_version: "encryptionkey@0"
I20260927 18:39:15.749878 17404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:15.763252 17404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:15.763561 17404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:15.763717 17404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:15.768692 17404 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:35547
I20260927 18:39:15.768782 17470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:35547 every 8 connection(s)
I20260927 18:39:15.769120 17404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:39:15.769294 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 17404
I20260927 18:39:15.769408 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:39:15.769675 12271 external_mini_cluster.cc:1428] Setting key 862239d073d42333d2409dd46a27cda2
I20260927 18:39:15.770498 17471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:15.774042 17471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89: Bootstrap starting.
I20260927 18:39:15.774889 17471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:15.775213 17471 log.cc:824] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:15.776125 17471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89: No bootstrap required, opened a new log
I20260927 18:39:15.776355 17474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:15.770266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37968 (local address 127.11.251.254:35547)
0927 18:39:15.770565 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:15.770575 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:15.770612 (+ 37us) server_negotiation.cc:407] Connection header received
0927 18:39:15.771241 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:15.771251 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:15.771337 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:15.771512 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:15.771698 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:15.773116 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:15.774042 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:15.774679 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:15.774815 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:15.775549 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:15.775552 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:15.775553 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:15.775737 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:15.775801 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:15.775816 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:15.775889 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:15.775980 (+ 91us) server_negotiation.cc:299] Negotiation successful
0927 18:39:15.776065 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":146,"threads_started":1}
I20260927 18:39:15.777458 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:39:15.778570 17471 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6cc9bab4fe4ec1955e4f7f6374ef89" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 35547 } }
I20260927 18:39:15.778748 17471 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:15.778776 17471 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac6cc9bab4fe4ec1955e4f7f6374ef89, State: Initialized, Role: FOLLOWER
I20260927 18:39:15.778923 17471 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6cc9bab4fe4ec1955e4f7f6374ef89" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 35547 } }
I20260927 18:39:15.779003 17471 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:39:15.779027 17471 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:39:15.779096 17471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:15.779683 17471 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6cc9bab4fe4ec1955e4f7f6374ef89" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 35547 } }
I20260927 18:39:15.779774 17471 leader_election.cc:302] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac6cc9bab4fe4ec1955e4f7f6374ef89; no voters:
I20260927 18:39:15.779953 17471 leader_election.cc:288] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:39:15.780098 17476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:15.780296 17476 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89 [term 1 LEADER]: Becoming Leader. State: Replica: ac6cc9bab4fe4ec1955e4f7f6374ef89, State: Running, Role: LEADER
I20260927 18:39:15.780298 17471 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:15.780435 17476 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6cc9bab4fe4ec1955e4f7f6374ef89" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 35547 } }
I20260927 18:39:15.780959 17477 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac6cc9bab4fe4ec1955e4f7f6374ef89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6cc9bab4fe4ec1955e4f7f6374ef89" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 35547 } } }
I20260927 18:39:15.781101 17477 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:15.780948 17478 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac6cc9bab4fe4ec1955e4f7f6374ef89. Latest consensus state: current_term: 1 leader_uuid: "ac6cc9bab4fe4ec1955e4f7f6374ef89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6cc9bab4fe4ec1955e4f7f6374ef89" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 35547 } } }
I20260927 18:39:15.781378 17478 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:15.781890 17485 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:39:15.782527 17485 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:39:15.784616 17485 catalog_manager.cc:1347] Generated new cluster ID: c6c2bd6e2f5b42969881497034080b82
I20260927 18:39:15.784679 17485 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:39:15.828886 17485 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:39:15.829478 17485 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:39:15.836683 17485 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac6cc9bab4fe4ec1955e4f7f6374ef89: Generated new TSK 0
I20260927 18:39:15.836886 17485 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260927 18:39:15.859457 17501 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260927 18:39:15.859566 17501 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260927 18:39:15.859841 17474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:15.844149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37978 (local address 127.11.251.254:35547)
0927 18:39:15.844197 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:15.844199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:15.844348 (+ 149us) server_negotiation.cc:407] Connection header received
0927 18:39:15.844464 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:15.844464 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:15.844478 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:15.844574 (+ 96us) server_negotiation.cc:226] Negotiated authn=JWT
0927 18:39:15.844794 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:15.845297 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:15.846506 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:15.846707 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:15.846768 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:15.847008 (+ 240us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 18:39:15.859613 (+ 12605us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43637/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0927 18:39:15.859782 (+ 169us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37978: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43637/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":22}
W20260927 18:39:15.859905 17474 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37978: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43637/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260927 18:39:15.860466 17500 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:15.844027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:35547 (local address 127.0.0.1:37978)
0927 18:39:15.844258 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:15.844273 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:15.844362 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:15.844617 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:15.844618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:15.844620 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0927 18:39:15.844719 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:15.844722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:15.845391 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:15.845392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:15.846406 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:15.846408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:15.846802 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:15.846803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:15.846865 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:15.846889 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 18:39:15.860051 (+ 13162us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43637/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0927 18:39:15.860175 (+ 124us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.251.254:35547: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43637/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":99,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.11.251.254:35547: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43637/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260927 18:39:15.860939 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 17404
2026-09-27T18:39:15Z chronyd exiting
I20260927 18:39:15.898941 12271 test_util.cc:175] -----------------------------------------------
I20260927 18:39:15.899065 12271 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534329098323-12271-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (267 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260927 18:39:15.900012 12271 mini_oidc.cc:152] Starting JWKS server
I20260927 18:39:15.900794 12271 webserver.cc:460] Webserver started at https://127.0.0.1:38119/ using document root <none> and password file <none>
I20260927 18:39:15.900862 12271 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 18:39:15.901055 12271 webserver.cc:460] Webserver started at http://127.0.0.1:44457/ using document root <none> and password file <none>
2026-09-27T18:39:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:39:15Z Disabled control of system clock
I20260927 18:39:15.910566 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:46731
--webserver_interface=127.11.251.254
--webserver_port=0
--builtin_ntp_servers=127.11.251.212:42967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:46731
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38119/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534329098323-12271-0/testchainca.cert with env {}
W20260927 18:39:15.997969 17507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:15.998212 17507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:15.998235 17507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:15.999502 17507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:15.999545 17507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:15.999558 17507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:15.999571 17507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:15.999629 17507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 18:39:15.999646 17507 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 18:39:15.999673 17507 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38119/jwks.json
I20260927 18:39:16.001330 17507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:42967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:46731
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:46731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38119/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534329098323-12271-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:16.001664 17507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:16.001941 17507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:16.007318 17515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:16.007303 17513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:16.007306 17512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:16.007642 17507 server_base.cc:1034] running on GCE node
I20260927 18:39:16.007897 17507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:16.008172 17507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:16.009338 17507 hybrid_clock.cc:648] HybridClock initialized: now 1790534356009304 us; error 47 us; skew 500 ppm
I20260927 18:39:16.010799 17507 webserver.cc:460] Webserver started at http://127.11.251.254:42587/ using document root <none> and password file <none>
I20260927 18:39:16.011083 17507 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:16.011161 17507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:16.011323 17507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:16.012616 17507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "2f5fc026afde49e9bf2edf7954420ebc"
format_stamp: "Formatted at 2026-09-27 18:39:16 on dist-test-slave-z47j"
server_key: "cebd1a8aadd3b1844fab82246ae3e298"
server_key_iv: "cfe74a818f6f322bce09133af9be6132"
server_key_version: "encryptionkey@0"
I20260927 18:39:16.013016 17507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "2f5fc026afde49e9bf2edf7954420ebc"
format_stamp: "Formatted at 2026-09-27 18:39:16 on dist-test-slave-z47j"
server_key: "cebd1a8aadd3b1844fab82246ae3e298"
server_key_iv: "cfe74a818f6f322bce09133af9be6132"
server_key_version: "encryptionkey@0"
I20260927 18:39:16.014496 17507 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:39:16.015343 17521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:16.015518 17507 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:39:16.015633 17507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "2f5fc026afde49e9bf2edf7954420ebc"
format_stamp: "Formatted at 2026-09-27 18:39:16 on dist-test-slave-z47j"
server_key: "cebd1a8aadd3b1844fab82246ae3e298"
server_key_iv: "cfe74a818f6f322bce09133af9be6132"
server_key_version: "encryptionkey@0"
I20260927 18:39:16.015695 17507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:16.038978 17507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:16.039306 17507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:16.039425 17507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:16.044346 17507 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:46731
I20260927 18:39:16.044441 17573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:46731 every 8 connection(s)
I20260927 18:39:16.044799 17507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:39:16.045694 17574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:16.045820 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 17507
I20260927 18:39:16.045969 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:39:16.046198 12271 external_mini_cluster.cc:1428] Setting key e49730a087f99bae6581a80e40c9c8b2
I20260927 18:39:16.050180 17574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc: Bootstrap starting.
I20260927 18:39:16.050848 17574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:16.051139 17574 log.cc:824] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:16.052068 17574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc: No bootstrap required, opened a new log
I20260927 18:39:16.053109 17577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:16.046846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42996 (local address 127.11.251.254:46731)
0927 18:39:16.047105 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:16.047116 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:16.047152 (+ 36us) server_negotiation.cc:407] Connection header received
0927 18:39:16.047809 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:16.047822 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:16.047921 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:16.048092 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:16.048463 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:16.049857 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:16.050760 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:16.051390 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:16.051506 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:16.052374 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:16.052380 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:16.052381 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:16.052593 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:16.052632 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:16.052646 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:16.052721 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:16.052799 (+ 78us) server_negotiation.cc:299] Negotiation successful
0927 18:39:16.052895 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":105,"threads_started":1}
I20260927 18:39:16.054302 17574 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5fc026afde49e9bf2edf7954420ebc" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46731 } }
I20260927 18:39:16.054502 17574 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:16.054533 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:39:16.054557 17574 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f5fc026afde49e9bf2edf7954420ebc, State: Initialized, Role: FOLLOWER
I20260927 18:39:16.054680 17574 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5fc026afde49e9bf2edf7954420ebc" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46731 } }
I20260927 18:39:16.054765 17574 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:39:16.054787 17574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:39:16.054857 17574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:16.055423 17574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5fc026afde49e9bf2edf7954420ebc" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46731 } }
I20260927 18:39:16.055533 17574 leader_election.cc:302] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f5fc026afde49e9bf2edf7954420ebc; no voters:
I20260927 18:39:16.055706 17574 leader_election.cc:288] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:39:16.055831 17579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:16.055965 17579 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc [term 1 LEADER]: Becoming Leader. State: Replica: 2f5fc026afde49e9bf2edf7954420ebc, State: Running, Role: LEADER
I20260927 18:39:16.056036 17574 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:16.056049 17579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5fc026afde49e9bf2edf7954420ebc" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46731 } }
I20260927 18:39:16.056453 17580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f5fc026afde49e9bf2edf7954420ebc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5fc026afde49e9bf2edf7954420ebc" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46731 } } }
I20260927 18:39:16.056484 17581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f5fc026afde49e9bf2edf7954420ebc. Latest consensus state: current_term: 1 leader_uuid: "2f5fc026afde49e9bf2edf7954420ebc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5fc026afde49e9bf2edf7954420ebc" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46731 } } }
I20260927 18:39:16.056577 17580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:16.056587 17581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:16.057020 17587 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:39:16.057611 17587 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:39:16.059206 17587 catalog_manager.cc:1347] Generated new cluster ID: 7bb4cbf9f96f4573a7e6ef47bffbce25
I20260927 18:39:16.059266 17587 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:39:16.072515 17587 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:39:16.073010 17587 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:39:16.076581 17587 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f5fc026afde49e9bf2edf7954420ebc: Generated new TSK 0
I20260927 18:39:16.076764 17587 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260927 18:39:16.100538 17604 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260927 18:39:16.100630 17604 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260927 18:39:16.101030 17577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:16.086048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43012 (local address 127.11.251.254:46731)
0927 18:39:16.086123 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:16.086126 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:16.086236 (+ 110us) server_negotiation.cc:407] Connection header received
0927 18:39:16.086313 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:16.086314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:16.086330 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:16.086421 (+ 91us) server_negotiation.cc:226] Negotiated authn=JWT
0927 18:39:16.086573 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:16.087410 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:16.088230 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:16.088562 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:16.088656 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:16.088938 (+ 282us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 18:39:16.100759 (+ 11821us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38119/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0927 18:39:16.100929 (+ 170us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43012: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38119/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":46}
W20260927 18:39:16.101060 17603 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:16.085947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:46731 (local address 127.0.0.1:43012)
0927 18:39:16.086153 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:16.086163 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:16.086242 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:16.086434 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:16.086435 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:16.086437 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0927 18:39:16.086502 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:16.086504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:16.087506 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:16.087507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:16.088140 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:16.088142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:16.088757 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:16.088759 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:16.088809 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:16.088815 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 18:39:16.100883 (+ 12068us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38119/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0927 18:39:16.100989 (+ 106us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.251.254:46731: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38119/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":127,"threads_started":1}
W20260927 18:39:16.101109 17577 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43012: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38119/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260927 18:39:16.101542 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 17507
2026-09-27T18:39:16Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (240 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260927 18:39:16.140410 12271 mini_oidc.cc:152] Starting JWKS server
I20260927 18:39:16.141101 12271 webserver.cc:460] Webserver started at https://127.0.0.1:36591/ using document root <none> and password file <none>
I20260927 18:39:16.141175 12271 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 18:39:16.141340 12271 webserver.cc:460] Webserver started at http://127.0.0.1:43305/ using document root <none> and password file <none>
2026-09-27T18:39:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:39:16Z Disabled control of system clock
I20260927 18:39:16.149405 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:35869
--webserver_interface=127.11.251.254
--webserver_port=0
--builtin_ntp_servers=127.11.251.212:38523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:35869
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36591/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260927 18:39:16.233758 17610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:16.233959 17610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:16.233981 17610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:16.235172 17610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:16.235215 17610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:16.235229 17610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:16.235240 17610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:16.235253 17610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 18:39:16.235311 17610 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 18:39:16.235327 17610 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36591/jwks.json
I20260927 18:39:16.236701 17610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:38523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:35869
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:35869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36591/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:16.236958 17610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:16.237289 17610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:16.242560 17618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:16.242550 17615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:16.242542 17616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:16.242826 17610 server_base.cc:1034] running on GCE node
I20260927 18:39:16.243083 17610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:16.243381 17610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:16.244567 17610 hybrid_clock.cc:648] HybridClock initialized: now 1790534356244540 us; error 47 us; skew 500 ppm
I20260927 18:39:16.245934 17610 webserver.cc:460] Webserver started at http://127.11.251.254:44911/ using document root <none> and password file <none>
I20260927 18:39:16.246196 17610 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:16.246266 17610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:16.246423 17610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:16.247684 17610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "7743e868beff4cf396072ca4d829ac0d"
format_stamp: "Formatted at 2026-09-27 18:39:16 on dist-test-slave-z47j"
server_key: "8590ba2f3005d10b8c47fd036b8a82c2"
server_key_iv: "b9911fb62bdbbb026aa618fdfb12b600"
server_key_version: "encryptionkey@0"
I20260927 18:39:16.248054 17610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "7743e868beff4cf396072ca4d829ac0d"
format_stamp: "Formatted at 2026-09-27 18:39:16 on dist-test-slave-z47j"
server_key: "8590ba2f3005d10b8c47fd036b8a82c2"
server_key_iv: "b9911fb62bdbbb026aa618fdfb12b600"
server_key_version: "encryptionkey@0"
I20260927 18:39:16.249449 17610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:39:16.250384 17624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:16.250605 17610 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:16.250716 17610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "7743e868beff4cf396072ca4d829ac0d"
format_stamp: "Formatted at 2026-09-27 18:39:16 on dist-test-slave-z47j"
server_key: "8590ba2f3005d10b8c47fd036b8a82c2"
server_key_iv: "b9911fb62bdbbb026aa618fdfb12b600"
server_key_version: "encryptionkey@0"
I20260927 18:39:16.250806 17610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:16.289755 17610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:16.290132 17610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:16.290295 17610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:16.294682 17610 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:35869
I20260927 18:39:16.294807 17676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:35869 every 8 connection(s)
I20260927 18:39:16.295115 17610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:39:16.295975 17677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:16.299188 17677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d: Bootstrap starting.
I20260927 18:39:16.299849 17677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:16.300181 17677 log.cc:824] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:16.300966 17677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d: No bootstrap required, opened a new log
I20260927 18:39:16.302688 17677 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7743e868beff4cf396072ca4d829ac0d" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 35869 } }
I20260927 18:39:16.302866 17677 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:16.302893 17677 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7743e868beff4cf396072ca4d829ac0d, State: Initialized, Role: FOLLOWER
I20260927 18:39:16.303086 17677 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7743e868beff4cf396072ca4d829ac0d" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 35869 } }
I20260927 18:39:16.303154 17677 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:39:16.303176 17677 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:39:16.303251 17677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:16.303838 17677 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7743e868beff4cf396072ca4d829ac0d" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 35869 } }
I20260927 18:39:16.303963 17677 leader_election.cc:302] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7743e868beff4cf396072ca4d829ac0d; no voters:
I20260927 18:39:16.304164 17677 leader_election.cc:288] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:39:16.304250 17680 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:16.304441 17680 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d [term 1 LEADER]: Becoming Leader. State: Replica: 7743e868beff4cf396072ca4d829ac0d, State: Running, Role: LEADER
I20260927 18:39:16.304467 17677 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:16.304618 17680 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7743e868beff4cf396072ca4d829ac0d" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 35869 } }
I20260927 18:39:16.304847 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 17610
I20260927 18:39:16.304957 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:39:16.305094 17681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7743e868beff4cf396072ca4d829ac0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7743e868beff4cf396072ca4d829ac0d" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 35869 } } }
I20260927 18:39:16.305183 17681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:16.305197 12271 external_mini_cluster.cc:1428] Setting key afba90051a2ffb21a66dd72941a0a8e8
I20260927 18:39:16.305464 17682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7743e868beff4cf396072ca4d829ac0d. Latest consensus state: current_term: 1 leader_uuid: "7743e868beff4cf396072ca4d829ac0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7743e868beff4cf396072ca4d829ac0d" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 35869 } } }
I20260927 18:39:16.305611 17682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:16.305627 17687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:39:16.306264 17687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:39:16.309067 17687 catalog_manager.cc:1347] Generated new cluster ID: 4f504b8814c64259b86bbd1d9f2fb365
I20260927 18:39:16.309129 17687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:39:16.312009 17691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:16.305769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54016 (local address 127.11.251.254:35869)
0927 18:39:16.306239 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:16.306252 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:16.306662 (+ 410us) server_negotiation.cc:407] Connection header received
0927 18:39:16.306802 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:16.306810 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:16.306890 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:16.307036 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:16.307489 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:16.308878 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:16.309840 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:16.310310 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:16.310473 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:16.311278 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:16.311282 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:16.311283 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:16.311489 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:16.311528 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:16.311539 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:16.311620 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:16.311695 (+ 75us) server_negotiation.cc:299] Negotiation successful
0927 18:39:16.311787 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":85,"threads_started":1}
I20260927 18:39:16.312896 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:39:16.331408 17687 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:39:16.331997 17687 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:39:16.335945 17687 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7743e868beff4cf396072ca4d829ac0d: Generated new TSK 0
I20260927 18:39:16.336139 17687 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:39:16.354691 17691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:16.350521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54028 (local address 127.11.251.254:35869)
0927 18:39:16.350580 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:16.350582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:16.350691 (+ 109us) server_negotiation.cc:407] Connection header received
0927 18:39:16.350747 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:16.350748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:16.350763 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:16.350886 (+ 123us) server_negotiation.cc:226] Negotiated authn=JWT
0927 18:39:16.351040 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:16.351870 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:16.352710 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:16.353042 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:16.353147 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:16.353372 (+ 225us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 18:39:16.354455 (+ 1083us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36591/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0927 18:39:16.354603 (+ 148us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54028: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36591/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":32}
W20260927 18:39:16.354820 17706 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:16.350420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:35869 (local address 127.0.0.1:54028)
0927 18:39:16.350595 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:16.350607 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:16.350683 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:16.350902 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:16.350902 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:16.350905 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0927 18:39:16.350969 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:16.350971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:16.351979 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:16.351979 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:16.352617 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:16.352618 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:16.353164 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:16.353165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:16.353237 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:16.353245 (+ 8us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 18:39:16.354563 (+ 1318us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36591/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0927 18:39:16.354700 (+ 137us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.251.254:35869: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36591/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":119,"threads_started":1}
I20260927 18:39:16.355165 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 17610
E20260927 18:39:16.366120 17707 webserver.cc:533] Webserver: sslize failed:
E20260927 18:39:16.366200 17707 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-27T18:39:16Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (252 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-27T18:39:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:39:16Z Disabled control of system clock
I20260927 18:39:16.402678 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:44253
--webserver_interface=127.11.251.254
--webserver_port=0
--builtin_ntp_servers=127.11.251.212:44701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:44253
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534329098323-12271-0/testchainca.cert with env {}
W20260927 18:39:16.490904 17711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:16.491144 17711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:16.491171 17711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:16.492482 17711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:16.492528 17711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:16.492542 17711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:16.492556 17711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:16.492568 17711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 18:39:16.492626 17711 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 18:39:16.492642 17711 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260927 18:39:16.494264 17711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:44701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:44253
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:44253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534329098323-12271-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:16.494567 17711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:16.494818 17711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:16.500646 17719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:16.500666 17717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:16.500646 17716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:16.500921 17711 server_base.cc:1034] running on GCE node
I20260927 18:39:16.501257 17711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:16.501700 17711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:16.502908 17711 hybrid_clock.cc:648] HybridClock initialized: now 1790534356502889 us; error 59 us; skew 500 ppm
I20260927 18:39:16.504460 17711 webserver.cc:460] Webserver started at http://127.11.251.254:34713/ using document root <none> and password file <none>
I20260927 18:39:16.504733 17711 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:16.504806 17711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:16.504966 17711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:16.506130 17711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "2411d3ada28a42b6b8bc90c07015e059"
format_stamp: "Formatted at 2026-09-27 18:39:16 on dist-test-slave-z47j"
server_key: "35fb35b7e37d8cd2058f22ae95f28152"
server_key_iv: "18559ce18660b94d377af84cdd24770e"
server_key_version: "encryptionkey@0"
I20260927 18:39:16.506500 17711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "2411d3ada28a42b6b8bc90c07015e059"
format_stamp: "Formatted at 2026-09-27 18:39:16 on dist-test-slave-z47j"
server_key: "35fb35b7e37d8cd2058f22ae95f28152"
server_key_iv: "18559ce18660b94d377af84cdd24770e"
server_key_version: "encryptionkey@0"
I20260927 18:39:16.508005 17711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:39:16.508882 17725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:16.509189 17711 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:16.509310 17711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "2411d3ada28a42b6b8bc90c07015e059"
format_stamp: "Formatted at 2026-09-27 18:39:16 on dist-test-slave-z47j"
server_key: "35fb35b7e37d8cd2058f22ae95f28152"
server_key_iv: "18559ce18660b94d377af84cdd24770e"
server_key_version: "encryptionkey@0"
I20260927 18:39:16.509390 17711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:16.528496 17711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:16.528827 17711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:16.528985 17711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:16.533555 17711 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:44253
I20260927 18:39:16.533704 17777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:44253 every 8 connection(s)
I20260927 18:39:16.534034 17711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:39:16.535192 17778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:16.537669 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 17711
I20260927 18:39:16.537773 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:39:16.537998 12271 external_mini_cluster.cc:1428] Setting key 1fd11f9dc957a6f82fa50884bfd8ab78
I20260927 18:39:16.538005 17778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059: Bootstrap starting.
I20260927 18:39:16.538904 17778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:16.539284 17778 log.cc:824] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:16.540153 17778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059: No bootstrap required, opened a new log
I20260927 18:39:16.542721 17778 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2411d3ada28a42b6b8bc90c07015e059" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44253 } }
I20260927 18:39:16.542907 17778 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:16.542940 17778 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2411d3ada28a42b6b8bc90c07015e059, State: Initialized, Role: FOLLOWER
I20260927 18:39:16.543078 17778 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2411d3ada28a42b6b8bc90c07015e059" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44253 } }
I20260927 18:39:16.543174 17778 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:39:16.543202 17778 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:39:16.543248 17778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:16.543967 17778 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2411d3ada28a42b6b8bc90c07015e059" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44253 } }
I20260927 18:39:16.544103 17778 leader_election.cc:302] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2411d3ada28a42b6b8bc90c07015e059; no voters:
I20260927 18:39:16.544283 17778 leader_election.cc:288] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:39:16.544345 17783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:16.544557 17783 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059 [term 1 LEADER]: Becoming Leader. State: Replica: 2411d3ada28a42b6b8bc90c07015e059, State: Running, Role: LEADER
I20260927 18:39:16.544600 17778 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:16.544703 17783 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2411d3ada28a42b6b8bc90c07015e059" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44253 } }
I20260927 18:39:16.545071 17783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2411d3ada28a42b6b8bc90c07015e059. Latest consensus state: current_term: 1 leader_uuid: "2411d3ada28a42b6b8bc90c07015e059" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2411d3ada28a42b6b8bc90c07015e059" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44253 } } }
I20260927 18:39:16.545158 17783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:16.545082 17785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2411d3ada28a42b6b8bc90c07015e059" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2411d3ada28a42b6b8bc90c07015e059" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44253 } } }
I20260927 18:39:16.545238 17785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:16.545430 17789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:39:16.545500 17781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:16.538655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49424 (local address 127.11.251.254:44253)
0927 18:39:16.538974 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:16.538984 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:16.539026 (+ 42us) server_negotiation.cc:407] Connection header received
0927 18:39:16.539715 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:16.539723 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:16.539822 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:16.540005 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:16.540349 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:16.541873 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:16.542748 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:16.543382 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:16.543521 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:16.544275 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:16.544279 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:16.544280 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:16.544534 (+ 254us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:16.544583 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:16.544605 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:16.544979 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:16.545020 (+ 41us) server_negotiation.cc:299] Negotiation successful
0927 18:39:16.545114 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":117,"threads_started":1}
I20260927 18:39:16.546599 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:39:16.546692 17789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:39:16.548429 17789 catalog_manager.cc:1347] Generated new cluster ID: 3744d707c96049e29dcb96fa1d95c0e6
I20260927 18:39:16.548480 17789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:39:16.565305 17789 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:39:16.565970 17789 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:39:16.578034 17789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2411d3ada28a42b6b8bc90c07015e059: Generated new TSK 0
I20260927 18:39:16.578289 17789 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:39:17.031665 17781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:16.589554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49428 (local address 127.11.251.254:44253)
0927 18:39:16.589651 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:16.589654 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:16.589778 (+ 124us) server_negotiation.cc:407] Connection header received
0927 18:39:16.589868 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:16.589868 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:16.589882 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:16.589947 (+ 65us) server_negotiation.cc:226] Negotiated authn=JWT
0927 18:39:16.590107 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:16.590629 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:16.591817 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:16.592015 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:16.592066 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:16.592258 (+ 192us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 18:39:17.031340 (+439082us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0927 18:39:17.031584 (+ 244us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49428: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":66}
W20260927 18:39:17.031778 17781 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49428: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260927 18:39:17.031812 17807 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:16.589490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:44253 (local address 127.0.0.1:49428)
0927 18:39:16.589704 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:16.589717 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:16.589793 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:16.589967 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:16.589968 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:16.589970 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0927 18:39:16.590040 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:16.590043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:16.590722 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:16.590723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:16.591734 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:16.591736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:16.592091 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:16.592092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:16.592156 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:16.592162 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 18:39:17.031528 (+439366us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0927 18:39:17.031700 (+ 172us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.251.254:44253: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":135,"threads_started":1}
I20260927 18:39:17.032271 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 17711
2026-09-27T18:39:17Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (658 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (100 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (97 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:39:17 dist-test-slave-z47j krb5kdc[17819](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:39:17 dist-test-slave-z47j krb5kdc[17819](info): set up 2 sockets
Sep 27 18:39:17 dist-test-slave-z47j krb5kdc[17819](info): commencing operation
krb5kdc: starting...
W20260927 18:39:18.274147 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:39:18 dist-test-slave-z47j krb5kdc[17819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:39:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:39:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:39:18.404595 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:40549
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:46159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:40549
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:18.493319 17832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:18.493552 17832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:18.493594 17832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:18.494841 17832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:18.494885 17832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:18.494900 17832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:18.494912 17832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:18.494925 17832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:39:18.496438 17832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:46159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:40549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:40549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:18.496717 17832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:18.496965 17832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:18.502586 17838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:18.502635 17837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:18.502737 17832 server_base.cc:1034] running on GCE node
W20260927 18:39:18.502641 17840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:18.503084 17832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:18.503443 17832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:18.504606 17832 hybrid_clock.cc:648] HybridClock initialized: now 1790534358504587 us; error 42 us; skew 500 ppm
Sep 27 18:39:18 dist-test-slave-z47j krb5kdc[17819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534358, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:18.511610 17832 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:39:18.512104 17832 webserver.cc:460] Webserver started at http://127.11.251.254:44119/ using document root <none> and password file <none>
I20260927 18:39:18.512375 17832 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:18.512431 17832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:18.512573 17832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:18.513840 17832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "db1714a01f2341c995c398ead55b04cc"
format_stamp: "Formatted at 2026-09-27 18:39:18 on dist-test-slave-z47j"
server_key: "6c2b2ae89beab045d472d936636a6143"
server_key_iv: "5995ac5b05822f37fb06edff6fef65b2"
server_key_version: "encryptionkey@0"
I20260927 18:39:18.514205 17832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "db1714a01f2341c995c398ead55b04cc"
format_stamp: "Formatted at 2026-09-27 18:39:18 on dist-test-slave-z47j"
server_key: "6c2b2ae89beab045d472d936636a6143"
server_key_iv: "5995ac5b05822f37fb06edff6fef65b2"
server_key_version: "encryptionkey@0"
I20260927 18:39:18.515645 17832 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:39:18.516532 17847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:18.516829 17832 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:18.516945 17832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "db1714a01f2341c995c398ead55b04cc"
format_stamp: "Formatted at 2026-09-27 18:39:18 on dist-test-slave-z47j"
server_key: "6c2b2ae89beab045d472d936636a6143"
server_key_iv: "5995ac5b05822f37fb06edff6fef65b2"
server_key_version: "encryptionkey@0"
I20260927 18:39:18.517024 17832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:18.532253 17832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:18.534245 17832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:18.534415 17832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:18.538991 17832 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:40549
I20260927 18:39:18.539093 17899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:40549 every 8 connection(s)
I20260927 18:39:18.539549 17832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:39:18.540144 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 17832
I20260927 18:39:18.540242 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:39:18.540464 12271 external_mini_cluster.cc:1428] Setting key 460100c2b1c09a6ffe58f31c49404b69
I20260927 18:39:18.540836 17900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:18.544677 17900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc: Bootstrap starting.
I20260927 18:39:18.545433 17900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:18.545835 17900 log.cc:824] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:18.546733 17900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc: No bootstrap required, opened a new log
Sep 27 18:39:18 dist-test-slave-z47j krb5kdc[17819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:18.549022 17900 raft_consensus.cc:348] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db1714a01f2341c995c398ead55b04cc" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40549 } }
I20260927 18:39:18.549176 17900 raft_consensus.cc:374] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:18.549229 17900 raft_consensus.cc:729] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db1714a01f2341c995c398ead55b04cc, State: Initialized, Role: FOLLOWER
I20260927 18:39:18.549393 17900 consensus_queue.cc:260] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db1714a01f2341c995c398ead55b04cc" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40549 } }
I20260927 18:39:18.549489 17900 raft_consensus.cc:388] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:39:18.549635 17900 raft_consensus.cc:482] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:39:18.549718 17900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:18.550395 17900 raft_consensus.cc:504] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db1714a01f2341c995c398ead55b04cc" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40549 } }
I20260927 18:39:18.550520 17900 leader_election.cc:302] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db1714a01f2341c995c398ead55b04cc; no voters:
I20260927 18:39:18.550769 17900 leader_election.cc:288] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:39:18.550802 17905 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:18.550958 17905 raft_consensus.cc:686] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc [term 1 LEADER]: Becoming Leader. State: Replica: db1714a01f2341c995c398ead55b04cc, State: Running, Role: LEADER
I20260927 18:39:18.551095 17900 sys_catalog.cc:564] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:18.551127 17905 consensus_queue.cc:237] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db1714a01f2341c995c398ead55b04cc" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40549 } }
I20260927 18:39:18.551546 17905 sys_catalog.cc:455] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader db1714a01f2341c995c398ead55b04cc. Latest consensus state: current_term: 1 leader_uuid: "db1714a01f2341c995c398ead55b04cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db1714a01f2341c995c398ead55b04cc" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40549 } } }
I20260927 18:39:18.551558 17906 sys_catalog.cc:455] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db1714a01f2341c995c398ead55b04cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db1714a01f2341c995c398ead55b04cc" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 40549 } } }
I20260927 18:39:18.551751 17905 sys_catalog.cc:458] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:18.552062 17906 sys_catalog.cc:458] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:18.552515 17903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:18.541068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59196 (local address 127.11.251.254:40549)
0927 18:39:18.541250 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:18.541258 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:18.541287 (+ 29us) server_negotiation.cc:407] Connection header received
0927 18:39:18.541791 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:18.541799 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:18.541847 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:18.541975 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:18.542875 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:18.543786 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:18.544901 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:18.545321 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:18.545408 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:18.548423 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:18.548427 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:18.548427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:18.548437 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:18.550433 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:18.550871 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:18.550875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:18.550875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:18.550987 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:18.551188 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:18.551189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:18.551189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:18.551508 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:18.551642 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:18.552116 (+ 474us) server_negotiation.cc:299] Negotiation successful
0927 18:39:18.552277 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
W20260927 18:39:18.553181 17920 catalog_manager.cc:1558] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:39:18.553414 17920 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:39:18.553507 17914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:39:18.553746 17914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:39:18.555505 17914 catalog_manager.cc:1347] Generated new cluster ID: bf738abec01b44c890015ca23cea62f4
I20260927 18:39:18.555565 17914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:39:18.574339 17914 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:39:18.574823 17914 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:39:18.578135 17914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db1714a01f2341c995c398ead55b04cc: Generated new TSK 0
I20260927 18:39:18.578353 17914 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.251.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
I20260927 18:39:18.612234 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:40549
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:46159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:18.703614 17926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:18.704110 17926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:18.704164 17926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:18.706060 17926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:18.706139 17926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:18.706199 17926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:39:18.708129 17926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:46159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:40549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:18.708465 17926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:18.708746 17926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:18.715497 17934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:18.715497 17932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:18.715536 17926 server_base.cc:1034] running on GCE node
W20260927 18:39:18.715569 17931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:18.716050 17926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:18.716442 17926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:18.717679 17926 hybrid_clock.cc:648] HybridClock initialized: now 1790534358717657 us; error 45 us; skew 500 ppm
Sep 27 18:39:18 dist-test-slave-z47j krb5kdc[17819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534358, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:18.721019 17926 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:39:18.721541 17926 webserver.cc:460] Webserver started at http://127.11.251.193:33633/ using document root <none> and password file <none>
I20260927 18:39:18.721822 17926 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:18.721869 17926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:18.721971 17926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:18.723112 17926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "789bd2216e9a431b8520eeb315976e68"
format_stamp: "Formatted at 2026-09-27 18:39:18 on dist-test-slave-z47j"
server_key: "7f79eb394f55b9c4f7e4b112319987f9"
server_key_iv: "e6a242bae1ca47a5c1ee140cd4adf744"
server_key_version: "encryptionkey@0"
I20260927 18:39:18.723438 17926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "789bd2216e9a431b8520eeb315976e68"
format_stamp: "Formatted at 2026-09-27 18:39:18 on dist-test-slave-z47j"
server_key: "7f79eb394f55b9c4f7e4b112319987f9"
server_key_iv: "e6a242bae1ca47a5c1ee140cd4adf744"
server_key_version: "encryptionkey@0"
I20260927 18:39:18.724859 17926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:39:18.725711 17941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:18.725906 17926 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:39:18.726002 17926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "789bd2216e9a431b8520eeb315976e68"
format_stamp: "Formatted at 2026-09-27 18:39:18 on dist-test-slave-z47j"
server_key: "7f79eb394f55b9c4f7e4b112319987f9"
server_key_iv: "e6a242bae1ca47a5c1ee140cd4adf744"
server_key_version: "encryptionkey@0"
I20260927 18:39:18.726063 17926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:18.741120 17926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:18.743355 17926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:18.743515 17926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:18.743731 17926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:18.744083 17926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:18.744117 17926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:18.744141 17926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:18.744194 17926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:18.750880 17926 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:39267
I20260927 18:39:18.750967 18054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:39267 every 8 connection(s)
I20260927 18:39:18.751338 17926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
Sep 27 18:39:18 dist-test-slave-z47j krb5kdc[17819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534358, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:18.758497 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 17926
I20260927 18:39:18.758623 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:39:18.758868 12271 external_mini_cluster.cc:1428] Setting key 5553c113657f93eeddce9b381bb3add3
I20260927 18:39:18.762488 17903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:18.752316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:38645 (local address 127.11.251.254:40549)
0927 18:39:18.752383 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:18.752386 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:18.752673 (+ 287us) server_negotiation.cc:407] Connection header received
0927 18:39:18.753863 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:18.753864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:18.753876 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:18.753936 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:18.754648 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:18.755273 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:18.756321 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:18.756559 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:18.756623 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:18.759173 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:18.759175 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:18.759175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:18.759183 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:18.760779 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:18.761252 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:18.761253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:18.761254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:18.761328 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:18.761609 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:18.761610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:18.761611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:18.761746 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:18.761825 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:18.762226 (+ 401us) server_negotiation.cc:299] Negotiation successful
0927 18:39:18.762339 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 18:39:18.762552 18057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:18.752292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:40549 (local address 127.11.251.193:38645)
0927 18:39:18.752477 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:18.752524 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:18.753749 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:18.754010 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:18.754015 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:18.754310 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:18.754535 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:18.754539 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:18.755366 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:18.755367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:18.756226 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:18.756229 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:18.756653 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:18.756654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:18.756758 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:18.757367 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:18.757377 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:18.759065 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:18.760900 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:18.760907 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:18.760910 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:18.761158 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:18.761399 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:18.761400 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:18.761400 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:18.761520 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:18.761814 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:18.761821 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:18.762038 (+ 217us) client_negotiation.cc:769] Sending connection context
0927 18:39:18.762185 (+ 147us) client_negotiation.cc:240] Negotiation successful
0927 18:39:18.762364 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":107,"threads_started":1}
I20260927 18:39:18.763571 18055 heartbeater.cc:347] Connected to a master server at 127.11.251.254:40549
I20260927 18:39:18.763686 18055 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:18.763897 18055 heartbeater.cc:511] Master 127.11.251.254:40549 requested a full tablet report, sending...
I20260927 18:39:18.764613 17864 ts_manager.cc:194] Registered new tserver with Master: 789bd2216e9a431b8520eeb315976e68 (127.11.251.193:39267)
I20260927 18:39:18.765686 17864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:38645
WARNING: no policy specified for kudu/127.11.251.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
I20260927 18:39:18.792865 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:40549
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:46159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:18.883167 18060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:18.883405 18060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:18.883432 18060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:18.884688 18060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:18.884737 18060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:18.884773 18060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:39:18.886372 18060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:46159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:40549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:18.886657 18060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:18.886946 18060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:39:18.892756 18060 server_base.cc:1034] running on GCE node
W20260927 18:39:18.892748 18065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:18.892868 18068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:18.892748 18066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:18.893179 18060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:18.893445 18060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:18.894629 18060 hybrid_clock.cc:648] HybridClock initialized: now 1790534358894616 us; error 48 us; skew 500 ppm
Sep 27 18:39:18 dist-test-slave-z47j krb5kdc[17819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534358, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:18.897313 18060 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:39:18.897853 18060 webserver.cc:460] Webserver started at http://127.11.251.194:45381/ using document root <none> and password file <none>
I20260927 18:39:18.898118 18060 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:18.898190 18060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:18.898341 18060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:18.899478 18060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "34753a2a3f5843f59ebf7e7fbe395b5a"
format_stamp: "Formatted at 2026-09-27 18:39:18 on dist-test-slave-z47j"
server_key: "4ef17e7d839da15e3551174d355988fb"
server_key_iv: "556d102d5afb75fe0004d5c75fb53de5"
server_key_version: "encryptionkey@0"
I20260927 18:39:18.899875 18060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "34753a2a3f5843f59ebf7e7fbe395b5a"
format_stamp: "Formatted at 2026-09-27 18:39:18 on dist-test-slave-z47j"
server_key: "4ef17e7d839da15e3551174d355988fb"
server_key_iv: "556d102d5afb75fe0004d5c75fb53de5"
server_key_version: "encryptionkey@0"
I20260927 18:39:18.901263 18060 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:39:18.902222 18075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:18.902397 18060 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:18.902499 18060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "34753a2a3f5843f59ebf7e7fbe395b5a"
format_stamp: "Formatted at 2026-09-27 18:39:18 on dist-test-slave-z47j"
server_key: "4ef17e7d839da15e3551174d355988fb"
server_key_iv: "556d102d5afb75fe0004d5c75fb53de5"
server_key_version: "encryptionkey@0"
I20260927 18:39:18.902611 18060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:18.917815 18060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:18.920135 18060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:18.920305 18060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:18.920594 18060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:18.920969 18060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:18.921056 18060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:18.921103 18060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:18.921118 18060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:18.927829 18060 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:41201
I20260927 18:39:18.927879 18188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:41201 every 8 connection(s)
I20260927 18:39:18.928231 18060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:39:18.929059 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 18060
I20260927 18:39:18.929177 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:39:18.929474 12271 external_mini_cluster.cc:1428] Setting key 64db5457a9b78b741f7b3d671f73a2d1
Sep 27 18:39:18 dist-test-slave-z47j krb5kdc[17819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534358, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.251.195@KRBTEST.COM; defaulting to no policy
I20260927 18:39:18.940203 17903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:18.929110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:36825 (local address 127.11.251.254:40549)
0927 18:39:18.929184 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:18.929188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:18.929412 (+ 224us) server_negotiation.cc:407] Connection header received
0927 18:39:18.930698 (+ 1286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:18.930700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:18.930716 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:18.930789 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:18.931441 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:18.932338 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:18.933985 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:18.934203 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:18.934277 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:18.936831 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:18.936833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:18.936834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:18.936843 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:18.938471 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:18.939029 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:18.939030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:18.939031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:18.939109 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:18.939443 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:18.939444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:18.939445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:18.939578 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:18.939666 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:18.939991 (+ 325us) server_negotiation.cc:299] Negotiation successful
0927 18:39:18.940090 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 18:39:18.940448 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:18.929072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:40549 (local address 127.11.251.194:36825)
0927 18:39:18.929253 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:18.929299 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:18.930596 (+ 1297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:18.930854 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:18.930857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:18.931140 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:18.931354 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:18.931358 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:18.932429 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:18.932431 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:18.933848 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:18.933851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:18.934295 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:18.934296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:18.934445 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:18.935120 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:18.935129 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:18.936725 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:18.938608 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:18.938615 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:18.938617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:18.938933 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:18.939222 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:18.939223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:18.939224 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:18.939349 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:18.939654 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:18.939661 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:18.939845 (+ 184us) client_negotiation.cc:769] Sending connection context
0927 18:39:18.939970 (+ 125us) client_negotiation.cc:240] Negotiation successful
0927 18:39:18.940155 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":104,"threads_started":1}
I20260927 18:39:18.941017 18189 heartbeater.cc:347] Connected to a master server at 127.11.251.254:40549
I20260927 18:39:18.941140 18189 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:18.941404 18189 heartbeater.cc:511] Master 127.11.251.254:40549 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.195@KRBTEST.COM" created.
I20260927 18:39:18.941941 17864 ts_manager.cc:194] Registered new tserver with Master: 34753a2a3f5843f59ebf7e7fbe395b5a (127.11.251.194:41201)
I20260927 18:39:18.942408 17864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:36825
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
I20260927 18:39:18.956830 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:40549
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:46159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:19.046128 18194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:19.046443 18194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:19.046521 18194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:19.047853 18194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:19.047907 18194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:19.047937 18194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:39:19.049589 18194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:46159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:40549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:19.049888 18194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:19.050181 18194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:19.056301 18200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:19.056295 18202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:19.056295 18199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:19.056632 18194 server_base.cc:1034] running on GCE node
I20260927 18:39:19.056900 18194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:19.057206 18194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:19.058444 18194 hybrid_clock.cc:648] HybridClock initialized: now 1790534359058415 us; error 70 us; skew 500 ppm
Sep 27 18:39:19 dist-test-slave-z47j krb5kdc[17819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534359, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:19.061470 18194 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:39:19.062060 18194 webserver.cc:460] Webserver started at http://127.11.251.195:38505/ using document root <none> and password file <none>
I20260927 18:39:19.062314 18194 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:19.062381 18194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:19.062551 18194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:19.063748 18194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "84ba020a24734e66987918d9be51e4f3"
format_stamp: "Formatted at 2026-09-27 18:39:19 on dist-test-slave-z47j"
server_key: "b70d32e62a7530f07bd1e9206520e57c"
server_key_iv: "8d29c8cfce9bd1d35f8c04b0f11c9d16"
server_key_version: "encryptionkey@0"
I20260927 18:39:19.064172 18194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "84ba020a24734e66987918d9be51e4f3"
format_stamp: "Formatted at 2026-09-27 18:39:19 on dist-test-slave-z47j"
server_key: "b70d32e62a7530f07bd1e9206520e57c"
server_key_iv: "8d29c8cfce9bd1d35f8c04b0f11c9d16"
server_key_version: "encryptionkey@0"
I20260927 18:39:19.065722 18194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:39:19.066624 18209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:19.066824 18194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:19.066951 18194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "84ba020a24734e66987918d9be51e4f3"
format_stamp: "Formatted at 2026-09-27 18:39:19 on dist-test-slave-z47j"
server_key: "b70d32e62a7530f07bd1e9206520e57c"
server_key_iv: "8d29c8cfce9bd1d35f8c04b0f11c9d16"
server_key_version: "encryptionkey@0"
I20260927 18:39:19.067032 18194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:19.085675 18194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:19.088261 18194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:19.088529 18194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:19.088891 18194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:19.089284 18194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:19.089320 18194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:19.089347 18194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:19.089360 18194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:19.102849 18194 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:39503
I20260927 18:39:19.102901 18322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:39503 every 8 connection(s)
I20260927 18:39:19.103694 18194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
Sep 27 18:39:19 dist-test-slave-z47j krb5kdc[17819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534359, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:19.113054 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 18194
I20260927 18:39:19.113165 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:39:19.113368 12271 external_mini_cluster.cc:1428] Setting key 9d2718cc005f1ada51fbc30a4f0acf56
I20260927 18:39:19.115787 17903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.104808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:48749 (local address 127.11.251.254:40549)
0927 18:39:19.104883 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:19.104886 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:19.105309 (+ 423us) server_negotiation.cc:407] Connection header received
0927 18:39:19.106423 (+ 1114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:19.106425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:19.106448 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:19.106508 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:19.107309 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.107872 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.109070 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.109283 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.109348 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:19.112200 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:19.112203 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:19.112203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:19.112215 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:19.114288 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:19.114760 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:19.114761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:19.114762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:19.114829 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:19.115088 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:19.115089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:19.115090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:19.115224 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:19.115304 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:19.115589 (+ 285us) server_negotiation.cc:299] Negotiation successful
0927 18:39:19.115688 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:39:19.116048 18325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.104711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:40549 (local address 127.11.251.195:48749)
0927 18:39:19.104971 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:19.105037 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:19.106246 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:19.106622 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:19.106627 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:19.106946 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:19.107208 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:19.107212 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.107986 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.107988 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:19.108975 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:19.108979 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.109381 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.109382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:19.109514 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:19.110187 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:19.110197 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:19.112016 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:19.114380 (+ 2364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:19.114387 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:19.114390 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:19.114667 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:19.114906 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:19.114906 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:19.114907 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:19.115021 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:19.115293 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:19.115299 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:19.115451 (+ 152us) client_negotiation.cc:769] Sending connection context
0927 18:39:19.115568 (+ 117us) client_negotiation.cc:240] Negotiation successful
0927 18:39:19.115750 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":99,"threads_started":1}
I20260927 18:39:19.116561 18323 heartbeater.cc:347] Connected to a master server at 127.11.251.254:40549
I20260927 18:39:19.116655 18323 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:19.116817 18323 heartbeater.cc:511] Master 127.11.251.254:40549 requested a full tablet report, sending...
I20260927 18:39:19.117206 17864 ts_manager.cc:194] Registered new tserver with Master: 84ba020a24734e66987918d9be51e4f3 (127.11.251.195:39503)
I20260927 18:39:19.117565 17864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:48749
I20260927 18:39:19.125811 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/test-admin.keytab.
Sep 27 18:39:19 dist-test-slave-z47j krb5kdc[17819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:19.143525 12271 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260927 18:39:19.145493 18330 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:19.144535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53786 (local address 127.11.251.193:39267)
0927 18:39:19.144775 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:19.144780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:19.144793 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:39:19.144827 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:19.144829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:19.144847 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:19.144934 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:19.145371 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53786: BlockingRecv error: recv got EOF from 127.0.0.1:53786 (error 108)
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":121,"threads_started":1}
W20260927 18:39:19.145711 12271 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/krb5cc' of type FILE
Sep 27 18:39:19 dist-test-slave-z47j krb5kdc[17819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:19.146793 12271 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 18:39:19 dist-test-slave-z47j krb5kdc[17819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:19.147857 12271 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 18:39:19 dist-test-slave-z47j krb5kdc[17819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.193@KRBTEST.COM
I20260927 18:39:19.157452 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.148441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53796 (local address 127.11.251.193:39267)
0927 18:39:19.148497 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:19.148500 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:19.148667 (+ 167us) server_negotiation.cc:407] Connection header received
0927 18:39:19.148700 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:19.148700 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:19.148713 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:19.148771 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:19.149196 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.150125 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.150922 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.151390 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.151453 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:19.154275 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:19.154285 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:19.154286 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:19.154294 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:19.156205 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:19.156557 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:19.156561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:19.156561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:19.156620 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:19.156786 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:19.156787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:19.156787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:19.156942 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:19.157200 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:19.157275 (+ 75us) server_negotiation.cc:299] Negotiation successful
0927 18:39:19.157355 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260927 18:39:19.160043 18330 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:19.159371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53800 (local address 127.11.251.193:39267)
0927 18:39:19.159438 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:19.159440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:19.159507 (+ 67us) server_negotiation.cc:407] Connection header received
0927 18:39:19.159580 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:19.159581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:19.159591 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:19.159632 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:19.159940 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53800: BlockingRecv error: recv got EOF from 127.0.0.1:53800 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260927 18:39:19.160290 12271 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/krb5cc' of type FILE
Sep 27 18:39:19 dist-test-slave-z47j krb5kdc[17819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:19.161630 12271 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 18:39:19 dist-test-slave-z47j krb5kdc[17819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:19.162649 12271 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 18:39:19 dist-test-slave-z47j krb5kdc[17819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.193@KRBTEST.COM
I20260927 18:39:19.171124 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.163495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53806 (local address 127.11.251.193:39267)
0927 18:39:19.163687 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:19.163690 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:19.163777 (+ 87us) server_negotiation.cc:407] Connection header received
0927 18:39:19.163863 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:19.163863 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:19.163877 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:19.163943 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:19.164362 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.164899 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.165724 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.165943 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.166056 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:19.168517 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:19.168519 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:19.168519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:19.168526 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:19.169880 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:19.170288 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:19.170289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:19.170289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:19.170350 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:19.170563 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:19.170563 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:19.170564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:19.170678 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:19.170759 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:19.170876 (+ 117us) server_negotiation.cc:299] Negotiation successful
0927 18:39:19.170948 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
W20260927 18:39:19.172791 18330 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:19.172262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53818 (local address 127.11.251.193:39267)
0927 18:39:19.172343 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:19.172344 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:19.172352 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:39:19.172415 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:19.172416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:19.172424 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:19.172474 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:19.172725 (+ 251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53818: BlockingRecv error: recv got EOF from 127.0.0.1:53818 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260927 18:39:19.172940 12271 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534329098323-12271-0/krb5kdc/krb5cc)
Sep 27 18:39:19 dist-test-slave-z47j krb5kdc[17819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:19.174189 12271 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 18:39:19 dist-test-slave-z47j krb5kdc[17819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:19.175124 12271 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 18:39:19 dist-test-slave-z47j krb5kdc[17819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.193@KRBTEST.COM
I20260927 18:39:19.182910 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.175679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53822 (local address 127.11.251.193:39267)
0927 18:39:19.175746 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:19.175755 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:19.175763 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:39:19.175782 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:19.175783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:19.175790 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:19.175850 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:19.176326 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.177261 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.178073 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.178263 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.178331 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:19.180426 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:19.180427 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:19.180428 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:19.180434 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:19.181879 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:19.182257 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:19.182258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:19.182258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:19.182307 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:19.182469 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:19.182469 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:19.182470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:19.182560 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:19.182665 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:19.182782 (+ 117us) server_negotiation.cc:299] Negotiation successful
0927 18:39:19.182841 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 18:39:19.183621 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 17926
I20260927 18:39:19.190062 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 18060
I20260927 18:39:19.194456 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 18194
I20260927 18:39:19.198729 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 17832
2026-09-27T18:39:19Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1970 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:39:19 dist-test-slave-z47j krb5kdc[18343](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:39:19 dist-test-slave-z47j krb5kdc[18343](info): set up 2 sockets
Sep 27 18:39:19 dist-test-slave-z47j krb5kdc[18343](info): commencing operation
krb5kdc: starting...
W20260927 18:39:20.254374 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:39:20 dist-test-slave-z47j krb5kdc[18343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:39:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:39:20Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.254.keytab.
Entry for principal oryx/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.254.keytab.
I20260927 18:39:20.381108 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:34227
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.254.keytab
--principal=oryx/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:40723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:34227
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:20.470310 18356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:20.470578 18356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:20.470604 18356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:20.471782 18356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:20.471818 18356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:20.471833 18356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:20.471846 18356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:20.471858 18356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:39:20.473335 18356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:40723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:34227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.254.keytab
--principal=oryx/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:34227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:20.473569 18356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:20.473863 18356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:20.479228 18362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:20.479220 18361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:20.479310 18356 server_base.cc:1034] running on GCE node
W20260927 18:39:20.479225 18364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:20.479655 18356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:20.479959 18356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:20.481117 18356 hybrid_clock.cc:648] HybridClock initialized: now 1790534360481120 us; error 45 us; skew 500 ppm
Sep 27 18:39:20 dist-test-slave-z47j krb5kdc[18343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534360, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:20.483814 18356 init.cc:377] Logged in from keytab as oryx/127.11.251.254@KRBTEST.COM (short username oryx)
I20260927 18:39:20.484238 18356 webserver.cc:460] Webserver started at http://127.11.251.254:36735/ using document root <none> and password file <none>
I20260927 18:39:20.484499 18356 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:20.484546 18356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:20.484755 18356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:20.486140 18356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "350e6f81203343e4ae0976f7e03d387e"
format_stamp: "Formatted at 2026-09-27 18:39:20 on dist-test-slave-z47j"
server_key: "3dcb95d183441ac09558427e06dc755b"
server_key_iv: "3ac9890d6adae8d7a470267847ea37c6"
server_key_version: "encryptionkey@0"
I20260927 18:39:20.486686 18356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "350e6f81203343e4ae0976f7e03d387e"
format_stamp: "Formatted at 2026-09-27 18:39:20 on dist-test-slave-z47j"
server_key: "3dcb95d183441ac09558427e06dc755b"
server_key_iv: "3ac9890d6adae8d7a470267847ea37c6"
server_key_version: "encryptionkey@0"
I20260927 18:39:20.488157 18356 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:39:20.488950 18371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:20.489195 18356 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:20.489301 18356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "350e6f81203343e4ae0976f7e03d387e"
format_stamp: "Formatted at 2026-09-27 18:39:20 on dist-test-slave-z47j"
server_key: "3dcb95d183441ac09558427e06dc755b"
server_key_iv: "3ac9890d6adae8d7a470267847ea37c6"
server_key_version: "encryptionkey@0"
I20260927 18:39:20.489378 18356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:20.498474 18356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:20.500478 18356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:20.500626 18356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:20.505187 18356 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:34227
I20260927 18:39:20.505231 18423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:34227 every 8 connection(s)
I20260927 18:39:20.505728 18356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:39:20.506711 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 18356
I20260927 18:39:20.506976 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:39:20.506961 18424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:20.507206 12271 external_mini_cluster.cc:1428] Setting key 17e1bffba96e30eabf7268542cf65f71
I20260927 18:39:20.510356 18424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e: Bootstrap starting.
I20260927 18:39:20.511052 18424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:20.511372 18424 log.cc:824] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:20.512122 18424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e: No bootstrap required, opened a new log
Sep 27 18:39:20 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM
I20260927 18:39:20.514156 18424 raft_consensus.cc:348] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } }
I20260927 18:39:20.514324 18424 raft_consensus.cc:374] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:20.514372 18424 raft_consensus.cc:729] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 350e6f81203343e4ae0976f7e03d387e, State: Initialized, Role: FOLLOWER
I20260927 18:39:20.514518 18424 consensus_queue.cc:260] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } }
I20260927 18:39:20.514623 18424 raft_consensus.cc:388] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:39:20.514672 18424 raft_consensus.cc:482] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:39:20.514746 18424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:20.515481 18424 raft_consensus.cc:504] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } }
I20260927 18:39:20.515606 18424 leader_election.cc:302] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 350e6f81203343e4ae0976f7e03d387e; no voters:
I20260927 18:39:20.515829 18424 leader_election.cc:288] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:39:20.515867 18429 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:20.515990 18429 raft_consensus.cc:686] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [term 1 LEADER]: Becoming Leader. State: Replica: 350e6f81203343e4ae0976f7e03d387e, State: Running, Role: LEADER
I20260927 18:39:20.516081 18429 consensus_queue.cc:237] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } }
I20260927 18:39:20.516140 18424 sys_catalog.cc:564] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:20.516499 18429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "350e6f81203343e4ae0976f7e03d387e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } }
I20260927 18:39:20.516602 18429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:20.516748 18429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 350e6f81203343e4ae0976f7e03d387e. Latest consensus state: current_term: 1 leader_uuid: "350e6f81203343e4ae0976f7e03d387e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } }
I20260927 18:39:20.516827 18429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [sys.catalog]: This master's current role is: LEADER
W20260927 18:39:20.518349 18445 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:39:20.518422 18445 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:39:20.518668 18427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:20.507823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53988 (local address 127.11.251.254:34227)
0927 18:39:20.508021 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:20.508033 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:20.508065 (+ 32us) server_negotiation.cc:407] Connection header received
0927 18:39:20.508596 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:20.508603 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:20.508653 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:20.508814 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:20.509489 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:20.510528 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:20.511320 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:20.511805 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:20.511940 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:20.514446 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:20.514450 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:20.514451 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:20.514462 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:20.516711 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:20.517167 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:20.517170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:20.517170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:20.517287 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:20.517527 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:20.517528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:20.517528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:20.517914 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:20.518065 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:20.518261 (+ 196us) server_negotiation.cc:299] Negotiation successful
0927 18:39:20.518409 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":89,"threads_started":1}
I20260927 18:39:20.518689 18435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:39:20.518903 18435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:39:20.520648 18435 catalog_manager.cc:1347] Generated new cluster ID: 08175ddff41c4f508594ea58d66b74dc
I20260927 18:39:20.520691 18435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:39:20.546319 18435 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:39:20.546941 18435 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:39:20.553489 18435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e: Generated new TSK 0
I20260927 18:39:20.553752 18435 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.251.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.251.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.251.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.193.keytab.
Entry for principal oryx/127.11.251.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.193.keytab.
I20260927 18:39:20.588881 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:34227
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.193.keytab
--principal=oryx/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:40723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:20.683224 18450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:20.683456 18450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:20.683481 18450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:20.684703 18450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:20.684741 18450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:20.684767 18450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:39:20.686280 18450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:40723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.193.keytab
--principal=oryx/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:34227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:20.686563 18450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:20.686825 18450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:20.692408 18455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:20.692431 18456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:20.692478 18458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:20.692749 18450 server_base.cc:1034] running on GCE node
I20260927 18:39:20.693005 18450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:20.693238 18450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:20.694409 18450 hybrid_clock.cc:648] HybridClock initialized: now 1790534360694396 us; error 33 us; skew 500 ppm
Sep 27 18:39:20 dist-test-slave-z47j krb5kdc[18343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534360, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:20.697113 18450 init.cc:377] Logged in from keytab as oryx/127.11.251.193@KRBTEST.COM (short username oryx)
I20260927 18:39:20.697659 18450 webserver.cc:460] Webserver started at http://127.11.251.193:45831/ using document root <none> and password file <none>
I20260927 18:39:20.697909 18450 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:20.697984 18450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:20.698159 18450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:20.699616 18450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "159be8c77cdc42d1b1a0840730dd6737"
format_stamp: "Formatted at 2026-09-27 18:39:20 on dist-test-slave-z47j"
server_key: "869d3d3daa8e26395de12eccc34da480"
server_key_iv: "b68ad690d55e6b32a6cd3c8eb57ebf0d"
server_key_version: "encryptionkey@0"
I20260927 18:39:20.700052 18450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "159be8c77cdc42d1b1a0840730dd6737"
format_stamp: "Formatted at 2026-09-27 18:39:20 on dist-test-slave-z47j"
server_key: "869d3d3daa8e26395de12eccc34da480"
server_key_iv: "b68ad690d55e6b32a6cd3c8eb57ebf0d"
server_key_version: "encryptionkey@0"
I20260927 18:39:20.701682 18450 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260927 18:39:20.702591 18465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:20.702772 18450 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:20.702883 18450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "159be8c77cdc42d1b1a0840730dd6737"
format_stamp: "Formatted at 2026-09-27 18:39:20 on dist-test-slave-z47j"
server_key: "869d3d3daa8e26395de12eccc34da480"
server_key_iv: "b68ad690d55e6b32a6cd3c8eb57ebf0d"
server_key_version: "encryptionkey@0"
I20260927 18:39:20.702977 18450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:20.727313 18450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:20.729399 18450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:20.729620 18450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:20.729905 18450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:20.730289 18450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:20.730356 18450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:20.730400 18450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:20.730458 18450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:20.737258 18450 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:36183
I20260927 18:39:20.737325 18578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:36183 every 8 connection(s)
I20260927 18:39:20.737721 18450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
Sep 27 18:39:20 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534360, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM
I20260927 18:39:20.745509 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 18450
I20260927 18:39:20.745651 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:39:20.745940 12271 external_mini_cluster.cc:1428] Setting key acb7171780a40c1377cb04e6e9678eaa
I20260927 18:39:20.749034 18427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:20.738647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:53597 (local address 127.11.251.254:34227)
0927 18:39:20.738741 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:20.738743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:20.739030 (+ 287us) server_negotiation.cc:407] Connection header received
0927 18:39:20.739603 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:20.739604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:20.739615 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:20.739681 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:20.740836 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:20.741402 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:20.742600 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:20.742846 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:20.742914 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:20.745421 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:20.745423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:20.745423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:20.745433 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:20.747461 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:20.747879 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:20.747881 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:20.747881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:20.747951 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:20.748195 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:20.748196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:20.748196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:20.748354 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:20.748425 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:20.748878 (+ 453us) server_negotiation.cc:299] Negotiation successful
0927 18:39:20.748957 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260927 18:39:20.749394 18581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:20.738604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.193:53597)
0927 18:39:20.738831 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:20.738876 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:20.739503 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:20.739772 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:20.739776 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:20.740498 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:20.740731 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:20.740736 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:20.741501 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:20.741502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:20.742471 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:20.742474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:20.742957 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:20.742958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:20.743076 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:20.743705 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:20.743716 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:20.745320 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:20.747555 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:20.747562 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:20.747564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:20.747806 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:20.748021 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:20.748022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:20.748022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:20.748128 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:20.748425 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:20.748431 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:20.748706 (+ 275us) client_negotiation.cc:769] Sending connection context
0927 18:39:20.748858 (+ 152us) client_negotiation.cc:240] Negotiation successful
0927 18:39:20.749067 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":104,"threads_started":1}
I20260927 18:39:20.750075 18579 heartbeater.cc:347] Connected to a master server at 127.11.251.254:34227
I20260927 18:39:20.750221 18579 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:20.750440 18579 heartbeater.cc:511] Master 127.11.251.254:34227 requested a full tablet report, sending...
I20260927 18:39:20.751092 18388 ts_manager.cc:194] Registered new tserver with Master: 159be8c77cdc42d1b1a0840730dd6737 (127.11.251.193:36183)
I20260927 18:39:20.751932 18388 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:53597
WARNING: no policy specified for oryx/127.11.251.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.251.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.251.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.194.keytab.
Entry for principal oryx/127.11.251.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.194.keytab.
I20260927 18:39:20.773093 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:34227
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.194.keytab
--principal=oryx/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:40723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:20.862646 18584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:20.862866 18584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:20.862890 18584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:20.864091 18584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:20.864132 18584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:20.864158 18584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:39:20.865711 18584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:40723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.194.keytab
--principal=oryx/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:34227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:20.865975 18584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:20.866253 18584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:20.871590 18592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:20.871749 18584 server_base.cc:1034] running on GCE node
W20260927 18:39:20.871661 18589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:20.871644 18590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:20.872079 18584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:20.872345 18584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:20.873525 18584 hybrid_clock.cc:648] HybridClock initialized: now 1790534360873520 us; error 38 us; skew 500 ppm
Sep 27 18:39:20 dist-test-slave-z47j krb5kdc[18343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534360, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:20.876266 18584 init.cc:377] Logged in from keytab as oryx/127.11.251.194@KRBTEST.COM (short username oryx)
I20260927 18:39:20.876767 18584 webserver.cc:460] Webserver started at http://127.11.251.194:36161/ using document root <none> and password file <none>
I20260927 18:39:20.877031 18584 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:20.877103 18584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:20.877257 18584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:20.878724 18584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "310dc769e00b4c279134a9d9b0f8bfe9"
format_stamp: "Formatted at 2026-09-27 18:39:20 on dist-test-slave-z47j"
server_key: "96ea273a53e7b80526ce96cf76e1b1ff"
server_key_iv: "e5d9eb19dce19aeb8667fb6a4f4144ae"
server_key_version: "encryptionkey@0"
I20260927 18:39:20.879108 18584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "310dc769e00b4c279134a9d9b0f8bfe9"
format_stamp: "Formatted at 2026-09-27 18:39:20 on dist-test-slave-z47j"
server_key: "96ea273a53e7b80526ce96cf76e1b1ff"
server_key_iv: "e5d9eb19dce19aeb8667fb6a4f4144ae"
server_key_version: "encryptionkey@0"
I20260927 18:39:20.880632 18584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:39:20.881436 18599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:20.881668 18584 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:39:20.881774 18584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "310dc769e00b4c279134a9d9b0f8bfe9"
format_stamp: "Formatted at 2026-09-27 18:39:20 on dist-test-slave-z47j"
server_key: "96ea273a53e7b80526ce96cf76e1b1ff"
server_key_iv: "e5d9eb19dce19aeb8667fb6a4f4144ae"
server_key_version: "encryptionkey@0"
I20260927 18:39:20.881865 18584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:20.898643 18584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:20.900808 18584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:20.900985 18584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:20.901247 18584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:20.901641 18584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:20.901693 18584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:20.901738 18584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:20.901782 18584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:20.908434 18584 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:43837
I20260927 18:39:20.908485 18712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:43837 every 8 connection(s)
I20260927 18:39:20.908851 18584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
Sep 27 18:39:20 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534360, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM
I20260927 18:39:20.918915 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 18584
I20260927 18:39:20.919050 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:39:20.919080 18427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:20.909630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:34201 (local address 127.11.251.254:34227)
0927 18:39:20.909698 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:20.909701 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:20.909922 (+ 221us) server_negotiation.cc:407] Connection header received
0927 18:39:20.910493 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:20.910494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:20.910541 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:20.910598 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:20.911593 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:20.912128 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:20.913163 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:20.913375 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:20.913489 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:20.915924 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:20.915926 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:20.915926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:20.915933 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:20.917491 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:20.918079 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:20.918080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:20.918080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:20.918137 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:20.918382 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:20.918382 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:20.918383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:20.918496 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:20.918576 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:20.918853 (+ 277us) server_negotiation.cc:299] Negotiation successful
0927 18:39:20.918958 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 18:39:20.919230 18715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:20.909565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.194:34201)
0927 18:39:20.909793 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:20.909832 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:20.910405 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:20.910671 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:20.910674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:20.911288 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:20.911510 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:20.911515 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:20.912205 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:20.912205 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:20.913056 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:20.913058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:20.913499 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:20.913500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:20.913666 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:20.914175 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:20.914184 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:20.915818 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:20.917766 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:20.917771 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:20.917773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:20.918009 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:20.918201 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:20.918202 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:20.918202 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:20.918319 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:20.918572 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:20.918577 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:20.918706 (+ 129us) client_negotiation.cc:769] Sending connection context
0927 18:39:20.918791 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 18:39:20.918922 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":91,"threads_started":1}
I20260927 18:39:20.919294 12271 external_mini_cluster.cc:1428] Setting key bcc00d1079cd922f0ce4bce55ccb9bd5
I20260927 18:39:20.919895 18713 heartbeater.cc:347] Connected to a master server at 127.11.251.254:34227
I20260927 18:39:20.920032 18713 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:20.920265 18713 heartbeater.cc:511] Master 127.11.251.254:34227 requested a full tablet report, sending...
I20260927 18:39:20.920753 18387 ts_manager.cc:194] Registered new tserver with Master: 310dc769e00b4c279134a9d9b0f8bfe9 (127.11.251.194:43837)
I20260927 18:39:20.921332 18387 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:34201
WARNING: no policy specified for oryx/127.11.251.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.251.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.251.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.195.keytab.
Entry for principal oryx/127.11.251.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.195.keytab.
I20260927 18:39:20.949828 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:34227
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.195.keytab
--principal=oryx/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:40723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:21.039762 18718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:21.039992 18718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:21.040016 18718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:21.041208 18718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:21.041254 18718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:21.041280 18718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:39:21.042833 18718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:40723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.195.keytab
--principal=oryx/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:34227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:21.043112 18718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:21.043365 18718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:21.048694 18723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:21.048745 18724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:21.048741 18718 server_base.cc:1034] running on GCE node
W20260927 18:39:21.048713 18726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:21.049063 18718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:21.049319 18718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:21.050491 18718 hybrid_clock.cc:648] HybridClock initialized: now 1790534361050471 us; error 37 us; skew 500 ppm
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534361, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:21.053259 18718 init.cc:377] Logged in from keytab as oryx/127.11.251.195@KRBTEST.COM (short username oryx)
I20260927 18:39:21.053861 18718 webserver.cc:460] Webserver started at http://127.11.251.195:35595/ using document root <none> and password file <none>
I20260927 18:39:21.054133 18718 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:21.054204 18718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:21.054359 18718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:21.055501 18718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "be295e1ae1234a83bb005c34e6a7ad4e"
format_stamp: "Formatted at 2026-09-27 18:39:21 on dist-test-slave-z47j"
server_key: "1af185f8d75702abffdf034cd8f080e5"
server_key_iv: "07092d9a14b2de3020738646f53122d9"
server_key_version: "encryptionkey@0"
I20260927 18:39:21.055884 18718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "be295e1ae1234a83bb005c34e6a7ad4e"
format_stamp: "Formatted at 2026-09-27 18:39:21 on dist-test-slave-z47j"
server_key: "1af185f8d75702abffdf034cd8f080e5"
server_key_iv: "07092d9a14b2de3020738646f53122d9"
server_key_version: "encryptionkey@0"
I20260927 18:39:21.057354 18718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:39:21.058240 18733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:21.058414 18718 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:39:21.058516 18718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "be295e1ae1234a83bb005c34e6a7ad4e"
format_stamp: "Formatted at 2026-09-27 18:39:21 on dist-test-slave-z47j"
server_key: "1af185f8d75702abffdf034cd8f080e5"
server_key_iv: "07092d9a14b2de3020738646f53122d9"
server_key_version: "encryptionkey@0"
I20260927 18:39:21.058604 18718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:21.070268 18718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:21.072268 18718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:21.072438 18718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:21.072680 18718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:21.073042 18718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:21.073108 18718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:21.073151 18718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:21.073182 18718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:21.080145 18718 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:37335
I20260927 18:39:21.080225 18846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:37335 every 8 connection(s)
I20260927 18:39:21.080583 18718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:39:21.085716 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 18718
I20260927 18:39:21.085868 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:39:21.086092 12271 external_mini_cluster.cc:1428] Setting key 30dbafd2fd7d2881d5f52966f2daaacf
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534361, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM
I20260927 18:39:21.090958 18427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.081388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:40825 (local address 127.11.251.254:34227)
0927 18:39:21.081445 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.081448 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.081652 (+ 204us) server_negotiation.cc:407] Connection header received
0927 18:39:21.082286 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.082288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.082304 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.082371 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.083400 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.083949 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.085077 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.085304 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.085392 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.087970 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.087972 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.087973 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.087981 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.089394 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.089929 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.089930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.089931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.089995 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.090272 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.090273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.090273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.090396 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.090474 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.090759 (+ 285us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.090858 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 18:39:21.091136 18849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.081347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.195:40825)
0927 18:39:21.081491 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.081542 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.082172 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.082447 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.082451 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.083073 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:21.083289 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.083294 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.084052 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.084053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.084976 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.084978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.085402 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.085404 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.085549 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.086182 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:21.086194 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:21.087873 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:21.089488 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.089495 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.089497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.089812 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.090062 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.090063 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.090064 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.090196 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.090474 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:21.090481 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:21.090623 (+ 142us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.090725 (+ 102us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.090885 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
I20260927 18:39:21.091624 18847 heartbeater.cc:347] Connected to a master server at 127.11.251.254:34227
I20260927 18:39:21.091727 18847 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:21.091920 18847 heartbeater.cc:511] Master 127.11.251.254:34227 requested a full tablet report, sending...
I20260927 18:39:21.092336 18387 ts_manager.cc:194] Registered new tserver with Master: be295e1ae1234a83bb005c34e6a7ad4e (127.11.251.195:37335)
I20260927 18:39:21.092710 18387 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:40825
I20260927 18:39:21.097975 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.251.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.251.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.251.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.253.keytab.
Entry for principal oryx/127.11.251.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.253.keytab.
I20260927 18:39:21.124972 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.253:41041
--webserver_interface=127.11.251.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.253.keytab
--principal=oryx/127.11.251.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:40723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:34227,127.11.251.253:41041
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:21.212746 18852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:21.212977 18852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:21.213002 18852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:21.214121 18852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:21.214157 18852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:21.214171 18852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:21.214184 18852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:21.214196 18852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:39:21.215778 18852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:40723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:34227,127.11.251.253:41041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.253.keytab
--principal=oryx/127.11.251.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.253:41041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.251.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
W20260927 18:39:21.215888 18852 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.11.251.254:34227,127.11.251.253:41041'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260927 18:39:21.216076 18852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:21.216291 18852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:21.221449 18858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:21.221454 18857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:21.221702 18860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:21.221724 18852 server_base.cc:1034] running on GCE node
I20260927 18:39:21.222034 18852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:21.222355 18852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:21.223520 18852 hybrid_clock.cc:648] HybridClock initialized: now 1790534361223514 us; error 47 us; skew 500 ppm
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534361, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:21.226364 18852 init.cc:377] Logged in from keytab as oryx/127.11.251.253@KRBTEST.COM (short username oryx)
I20260927 18:39:21.226886 18852 webserver.cc:460] Webserver started at http://127.11.251.253:45249/ using document root <none> and password file <none>
I20260927 18:39:21.227156 18852 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:21.227217 18852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:21.227360 18852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:21.228456 18852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/data/instance:
uuid: "fda725da13614957a67aa50a78c8770e"
format_stamp: "Formatted at 2026-09-27 18:39:21 on dist-test-slave-z47j"
server_key: "482a7580fb9abfe4ccbce43878a99b7d"
server_key_iv: "cbd5484a4d8c7e14851de0285ead5201"
server_key_version: "encryptionkey@0"
I20260927 18:39:21.228782 18852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/wal/instance:
uuid: "fda725da13614957a67aa50a78c8770e"
format_stamp: "Formatted at 2026-09-27 18:39:21 on dist-test-slave-z47j"
server_key: "482a7580fb9abfe4ccbce43878a99b7d"
server_key_iv: "cbd5484a4d8c7e14851de0285ead5201"
server_key_version: "encryptionkey@0"
I20260927 18:39:21.230204 18852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:39:21.230929 18867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:21.231101 18852 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:39:21.231206 18852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/wal
uuid: "fda725da13614957a67aa50a78c8770e"
format_stamp: "Formatted at 2026-09-27 18:39:21 on dist-test-slave-z47j"
server_key: "482a7580fb9abfe4ccbce43878a99b7d"
server_key_iv: "cbd5484a4d8c7e14851de0285ead5201"
server_key_version: "encryptionkey@0"
I20260927 18:39:21.231282 18852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:21.265303 18852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:21.267915 18852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:21.268093 18852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:21.272753 18852 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.253:41041
I20260927 18:39:21.272789 18919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.253:41041 every 8 connection(s)
I20260927 18:39:21.273233 18852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/data/info.pb
I20260927 18:39:21.274407 18920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:21.276134 18920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:39:21.280488 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 18852
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534361, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.253@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM
I20260927 18:39:21.286576 18427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.276923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54002 (local address 127.11.251.254:34227)
0927 18:39:21.276995 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.276997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.277229 (+ 232us) server_negotiation.cc:407] Connection header received
0927 18:39:21.277842 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.277843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.277854 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.277922 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.278548 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.279162 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.280206 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.280508 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.280585 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.283208 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.283210 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.283210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.283217 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.284822 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.285390 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.285391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.285392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.285459 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.285763 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.285764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.285765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.286029 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.286136 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.286434 (+ 298us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.286501 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:39:21.286836 18922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.276884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.0.0.1:54002)
0927 18:39:21.277094 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.277145 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.277747 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.277981 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.277985 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.278260 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:21.278471 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.278475 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.279251 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.279252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.280132 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.280135 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.280607 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.280608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.280809 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.281545 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:21.281556 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:21.283112 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:21.284891 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.284898 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.284900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.285248 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.285540 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.285540 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.285540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.285692 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.286126 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:21.286131 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:21.286256 (+ 125us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.286389 (+ 133us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.286569 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":101,"threads_started":1}
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.251.253@KRBTEST.COM
I20260927 18:39:21.287549 18920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:39:21.290653 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.281045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50294 (local address 127.11.251.253:41041)
0927 18:39:21.281233 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.281238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.281347 (+ 109us) server_negotiation.cc:407] Connection header received
0927 18:39:21.281381 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.281383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.281405 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.281515 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.282239 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.282984 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.283921 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.284490 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.284579 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.287548 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.287552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.287553 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.287558 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.289322 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.289777 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.289780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.289780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.289844 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.290049 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.290050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.290050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.290210 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.290381 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.290462 (+ 81us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.290538 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":110,"threads_started":1}
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534361, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.253@KRBTEST.COM for oryx/127.11.251.253@KRBTEST.COM
I20260927 18:39:21.295184 18925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.287891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50308 (local address 127.11.251.253:41041)
0927 18:39:21.288112 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.288114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.288121 (+ 7us) server_negotiation.cc:407] Connection header received
0927 18:39:21.288136 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.288136 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.288143 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.288205 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.288535 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.289137 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.289862 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.290079 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.290170 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.292120 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.292121 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.292122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.292128 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.294062 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.294395 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.294396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.294396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.294456 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.294656 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.294657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.294658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.294776 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.294851 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.294968 (+ 117us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.295045 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":97,"threads_started":1}
I20260927 18:39:21.295186 18922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.287847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:41041 (local address 127.0.0.1:50308)
0927 18:39:21.287888 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.287904 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.287932 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.288231 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.288232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.288415 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:21.288471 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.288473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.289232 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.289233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.289776 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.289777 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.290159 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.290170 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.290234 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.290646 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:21.290651 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:21.292030 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:21.294156 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.294157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.294157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.294316 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.294536 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.294537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.294537 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.294592 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.294840 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:21.294841 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:21.294876 (+ 35us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.294954 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.295090 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":60}
I20260927 18:39:21.296725 18920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: Bootstrap starting.
I20260927 18:39:21.297472 18920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:21.297821 18920 log.cc:824] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:21.298583 18920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: No bootstrap required, opened a new log
I20260927 18:39:21.300117 18920 raft_consensus.cc:348] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } }
I20260927 18:39:21.300273 18920 raft_consensus.cc:374] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:21.300323 18920 raft_consensus.cc:729] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fda725da13614957a67aa50a78c8770e, State: Initialized, Role: FOLLOWER
I20260927 18:39:21.300468 18920 consensus_queue.cc:260] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } }
I20260927 18:39:21.300773 18927 sys_catalog.cc:455] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } } }
I20260927 18:39:21.300853 18920 sys_catalog.cc:564] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:21.300865 18927 sys_catalog.cc:458] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [sys.catalog]: This master's current role is: FOLLOWER
W20260927 18:39:21.303016 18938 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:39:21.303138 18938 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:39:21.304721 18852 master_runner.cc:428] Detected that this master fda725da13614957a67aa50a78c8770e is joining an existing cluster
I20260927 18:39:21.304793 18852 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.251.253:41041
I20260927 18:39:21.312738 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.305941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:41041 (local address 127.0.0.1:50312)
0927 18:39:21.306129 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.306141 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.306198 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.306308 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.306308 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.306497 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:21.306568 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.306570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.307312 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.307313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.307890 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.307891 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.308315 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.308316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.308390 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.308912 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:21.308916 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:21.309272 (+ 356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:21.311573 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.311574 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.311575 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.311750 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.311982 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.311983 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.311983 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.312057 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.312372 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:21.312373 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:21.312414 (+ 41us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.312495 (+ 81us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.312602 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":106,"threads_started":1}
I20260927 18:39:21.312933 18925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.306183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50312 (local address 127.11.251.253:41041)
0927 18:39:21.306219 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.306222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.306229 (+ 7us) server_negotiation.cc:407] Connection header received
0927 18:39:21.306248 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.306249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.306258 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.306306 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.306643 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.307229 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.307985 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.308197 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.308287 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.309377 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.309379 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.309379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.309385 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.311445 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.311834 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.311835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.311836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.311891 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.312116 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.312116 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.312117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.312286 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.312348 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.312517 (+ 169us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.312851 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 18:39:21.316059 18427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.306003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54008 (local address 127.11.251.254:34227)
0927 18:39:21.306064 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.306067 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.309794 (+ 3727us) server_negotiation.cc:407] Connection header received
0927 18:39:21.309856 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.309857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.309870 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.309903 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.310375 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.310895 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.311957 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.312248 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.312325 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.313432 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.313433 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.313434 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.313442 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.314923 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.315307 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.315308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.315308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.315364 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.315555 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.315555 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.315556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.315664 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.315716 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.315900 (+ 184us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.315976 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 18:39:21.316066 18945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.306042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.0.0.1:54008)
0927 18:39:21.309700 (+ 3658us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.309713 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.309786 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.309971 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.309972 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.310208 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:21.310285 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.310287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.310987 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.310989 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.311867 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.311870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.312333 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.312334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.312409 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.312808 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:21.312813 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:21.313186 (+ 373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:21.315000 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.315001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.315002 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.315224 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.315425 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.315426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.315426 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.315488 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.315727 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:21.315727 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:21.315771 (+ 44us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.315861 (+ 90us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.315956 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3638,"thread_start_us":88,"threads_started":1}
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534360, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.254@KRBTEST.COM for oryx/127.11.251.253@KRBTEST.COM
I20260927 18:39:21.326259 18925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.318720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50318 (local address 127.11.251.253:41041)
0927 18:39:21.318759 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.318761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.318862 (+ 101us) server_negotiation.cc:407] Connection header received
0927 18:39:21.318963 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.318963 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.318971 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.319012 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.319516 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.319952 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.321115 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.321294 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.321360 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.323610 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.323612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.323612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.323619 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.324953 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.325401 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.325402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.325403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.325459 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.325724 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.325725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.325725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.325828 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.325902 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.326089 (+ 187us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.326174 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:39:21.326300 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.318614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:41041 (local address 127.0.0.1:50318)
0927 18:39:21.318764 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.318807 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.318888 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.319028 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.319031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.319376 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:21.319444 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.319446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.320031 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.320032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.321024 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.321026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.321374 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.321375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.321460 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.321891 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:21.321896 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:21.323519 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:21.325090 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.325097 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.325100 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.325308 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.325533 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.325534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.325534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.325672 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.325894 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:21.325901 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:21.325954 (+ 53us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.326062 (+ 108us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.326183 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":97,"threads_started":1}
I20260927 18:39:21.326881 18387 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.251.253:41041: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "fda725da13614957a67aa50a78c8770e"
member_type: NON_VOTER
last_known_addr {
host: "127.11.251.253"
port: 41041
}
attrs {
promote: true
}
}
dest_uuid: "350e6f81203343e4ae0976f7e03d387e"
cas_config_opid_index: -1
I20260927 18:39:21.327265 18387 consensus_queue.cc:237] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: NON_VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: true } }
I20260927 18:39:21.327863 18948 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER fda725da13614957a67aa50a78c8770e (127.11.251.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: NON_VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: true } } }
I20260927 18:39:21.328171 18894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:21.329131 18948 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.251.253:41041
I20260927 18:39:21.329106 18894 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 1 FOLLOWER]: Refusing update from remote peer 350e6f81203343e4ae0976f7e03d387e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260927 18:39:21.329355 18852 master.cc:537] Master@127.11.251.253:41041 shutting down...
I20260927 18:39:21.329532 18947 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: NON_VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:21.330133 18949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "350e6f81203343e4ae0976f7e03d387e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: NON_VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: true } } }
I20260927 18:39:21.330229 18949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [sys.catalog]: This master's current role is: LEADER
W20260927 18:39:21.330883 18374 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260927 18:39:21.331429 18374 consensus_peers.cc:487] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e -> Peer fda725da13614957a67aa50a78c8770e (127.11.251.253:41041): Couldn't send request to peer fda725da13614957a67aa50a78c8770e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260927 18:39:21.332501 18852 raft_consensus.cc:2229] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:21.332610 18852 raft_consensus.cc:2258] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:21.332669 18852 tablet_replica.cc:330] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: stopping tablet replica
I20260927 18:39:21.345685 18852 master.cc:552] Master@127.11.251.253:41041 shutdown complete.
I20260927 18:39:21.345769 18852 master_runner.cc:305] Clearing existing system tablet
I20260927 18:39:21.345925 18852 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:21.345990 18852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:21.346668 18852 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:39:21.347215 18953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:21.347329 18852 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:21.347390 18852 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:21.347412 18852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/wal
uuid: "fda725da13614957a67aa50a78c8770e"
format_stamp: "Formatted at 2026-09-27 18:39:21 on dist-test-slave-z47j"
server_key: "482a7580fb9abfe4ccbce43878a99b7d"
server_key_iv: "cbd5484a4d8c7e14851de0285ead5201"
server_key_version: "encryptionkey@0"
I20260927 18:39:21.348151 18852 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:21.349795 18852 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260927 18:39:21.349884 18852 log.cc:1196] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260927 18:39:21.350091 18852 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: Deleting consensus metadata
I20260927 18:39:21.350251 18852 master_runner.cc:315] Copying system tablet from 127.11.251.254:34227
I20260927 18:39:21.350937 18852 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: tablet copy: Beginning tablet copy session from remote peer at address 127.11.251.254:34227
I20260927 18:39:21.358006 18959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.351301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.0.0.1:54022)
0927 18:39:21.351450 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.351462 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.351532 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.351706 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.351707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.351882 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:21.351949 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.351951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.352995 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.352996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.353659 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.353662 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.354152 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.354153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.354211 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.354758 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:21.354763 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:21.355121 (+ 358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:21.356872 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.356873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.356874 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.357081 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.357310 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.357311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.357312 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.357373 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.357698 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:21.357700 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:21.357748 (+ 48us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.357812 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.357909 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260927 18:39:21.358053 18427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.351407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54022 (local address 127.11.251.254:34227)
0927 18:39:21.351491 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.351493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.351510 (+ 17us) server_negotiation.cc:407] Connection header received
0927 18:39:21.351610 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.351611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.351624 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.351703 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.352030 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.352897 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.353745 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.354062 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.354147 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.355208 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.355210 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.355210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.355219 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.356789 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.357160 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.357161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.357162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.357226 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.357449 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.357450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.357450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.357623 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.357698 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.357863 (+ 165us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.357949 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:39:21.358621 18408 tablet_copy_service.cc:140] P 350e6f81203343e4ae0976f7e03d387e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer fda725da13614957a67aa50a78c8770e ({username='oryx', principal='oryx/127.11.251.253@KRBTEST.COM'} at 127.0.0.1:54022)
I20260927 18:39:21.358747 18408 tablet_copy_service.cc:161] P 350e6f81203343e4ae0976f7e03d387e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer fda725da13614957a67aa50a78c8770e at {username='oryx', principal='oryx/127.11.251.253@KRBTEST.COM'} at 127.0.0.1:54022: session id = fda725da13614957a67aa50a78c8770e-00000000000000000000000000000000
I20260927 18:39:21.359899 18408 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e: Tablet Copy: opened 0 blocks and 1 log segments
I20260927 18:39:21.360747 18852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260927 18:39:21.362051 18852 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: tablet copy: Starting download of 0 data blocks...
I20260927 18:39:21.362226 18852 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: tablet copy: Starting download of 1 WAL segments...
I20260927 18:39:21.363165 18852 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260927 18:39:21.364045 18408 tablet_copy_service.cc:342] P 350e6f81203343e4ae0976f7e03d387e: Request end of tablet copy session fda725da13614957a67aa50a78c8770e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.251.253@KRBTEST.COM'} at 127.0.0.1:54022
I20260927 18:39:21.364130 18408 tablet_copy_service.cc:433] P 350e6f81203343e4ae0976f7e03d387e: ending tablet copy session fda725da13614957a67aa50a78c8770e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer fda725da13614957a67aa50a78c8770e
I20260927 18:39:21.366892 18852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:21.368188 18960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:21.368315 18963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:21.368467 18852 server_base.cc:1034] running on GCE node
W20260927 18:39:21.368502 18961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:21.368672 18852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:21.368871 18852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:21.370010 18852 hybrid_clock.cc:648] HybridClock initialized: now 1790534361369998 us; error 30 us; skew 500 ppm
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534361, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:21.371466 18852 init.cc:377] Logged in from keytab as oryx/127.11.251.253@KRBTEST.COM (short username oryx)
I20260927 18:39:21.371752 18852 webserver.cc:460] Webserver started at http://127.11.251.253:37629/ using document root <none> and password file <none>
I20260927 18:39:21.371858 18852 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:21.371886 18852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:21.372494 18852 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:21.372989 18970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:21.373111 18852 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:21.373159 18852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/wal
uuid: "fda725da13614957a67aa50a78c8770e"
format_stamp: "Formatted at 2026-09-27 18:39:21 on dist-test-slave-z47j"
server_key: "482a7580fb9abfe4ccbce43878a99b7d"
server_key_iv: "cbd5484a4d8c7e14851de0285ead5201"
server_key_version: "encryptionkey@0"
I20260927 18:39:21.373211 18852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:21.382421 18852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:21.384112 18852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:21.387717 18852 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.253:41041
I20260927 18:39:21.388051 19022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.253:41041 every 8 connection(s)
I20260927 18:39:21.388142 18852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-1/data/info.pb
I20260927 18:39:21.388976 19023 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:39:21.389460 19023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: Bootstrap starting.
I20260927 18:39:21.392509 19023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:39:21.392779 19023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: Bootstrap complete.
I20260927 18:39:21.393047 19023 raft_consensus.cc:348] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: NON_VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: true } }
I20260927 18:39:21.393151 19023 raft_consensus.cc:729] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: fda725da13614957a67aa50a78c8770e, State: Initialized, Role: LEARNER
I20260927 18:39:21.393203 19023 consensus_queue.cc:260] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: NON_VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: true } }
I20260927 18:39:21.393371 19027 sys_catalog.cc:455] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: NON_VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: true } } }
I20260927 18:39:21.393486 19027 sys_catalog.cc:458] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [sys.catalog]: This master's current role is: LEARNER
I20260927 18:39:21.393419 19023 sys_catalog.cc:564] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:21.395206 19038 catalog_manager.cc:1259] Loaded cluster ID: 08175ddff41c4f508594ea58d66b74dc
I20260927 18:39:21.395833 19038 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: loading cluster ID for follower catalog manager: success
I20260927 18:39:21.397125 19038 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: acquiring CA information for follower catalog manager: success
I20260927 18:39:21.397449 19038 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 18:39:21.402909 19026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.392652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50342 (local address 127.11.251.253:41041)
0927 18:39:21.393084 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.393087 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.393219 (+ 132us) server_negotiation.cc:407] Connection header received
0927 18:39:21.393496 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.393502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.393517 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.395714 (+ 2197us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.395758 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.396749 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.397847 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.398066 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.398122 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.399607 (+ 1485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.399609 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.399609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.399616 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.401102 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.401547 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.401549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.401549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.401654 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.401933 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.401935 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.401935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.402115 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.402275 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.402684 (+ 409us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.402788 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":185,"threads_started":1}
WARNING: no policy specified for oryx/127.11.251.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.251.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.251.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.252.keytab.
Entry for principal oryx/127.11.251.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.252.keytab.
I20260927 18:39:21.432279 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.252:39555
--webserver_interface=127.11.251.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.252.keytab
--principal=oryx/127.11.251.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:40723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:34227,127.11.251.253:41041,127.11.251.252:39555
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:21.524437 19041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:21.524691 19041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:21.524715 19041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:21.525899 19041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:21.525938 19041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:21.525954 19041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:21.525966 19041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:21.525978 19041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:39:21.527727 19041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:40723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:34227,127.11.251.253:41041,127.11.251.252:39555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.252.keytab
--principal=oryx/127.11.251.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.252:39555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.251.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:21.528031 19041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:21.528332 19041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:39:21.533757 19041 server_base.cc:1034] running on GCE node
W20260927 18:39:21.533732 19049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:21.533733 19046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:21.533740 19047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:21.534196 19041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:21.534595 19041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:21.535761 19041 hybrid_clock.cc:648] HybridClock initialized: now 1790534361535749 us; error 47 us; skew 500 ppm
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534361, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:21.538559 19041 init.cc:377] Logged in from keytab as oryx/127.11.251.252@KRBTEST.COM (short username oryx)
I20260927 18:39:21.539059 19041 webserver.cc:460] Webserver started at http://127.11.251.252:38109/ using document root <none> and password file <none>
I20260927 18:39:21.539319 19041 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:21.539392 19041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:21.539551 19041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:21.540737 19041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/data/instance:
uuid: "f5cb1a26dafa4b76ab1b5cadcab65903"
format_stamp: "Formatted at 2026-09-27 18:39:21 on dist-test-slave-z47j"
server_key: "837b6ef2295939ea066e4648315a62a7"
server_key_iv: "7527d510ed4e28886d49442389c917d3"
server_key_version: "encryptionkey@0"
I20260927 18:39:21.541141 19041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/wal/instance:
uuid: "f5cb1a26dafa4b76ab1b5cadcab65903"
format_stamp: "Formatted at 2026-09-27 18:39:21 on dist-test-slave-z47j"
server_key: "837b6ef2295939ea066e4648315a62a7"
server_key_iv: "7527d510ed4e28886d49442389c917d3"
server_key_version: "encryptionkey@0"
I20260927 18:39:21.542641 19041 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:39:21.543417 19056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:21.543606 19041 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:39:21.543718 19041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/wal
uuid: "f5cb1a26dafa4b76ab1b5cadcab65903"
format_stamp: "Formatted at 2026-09-27 18:39:21 on dist-test-slave-z47j"
server_key: "837b6ef2295939ea066e4648315a62a7"
server_key_iv: "7527d510ed4e28886d49442389c917d3"
server_key_version: "encryptionkey@0"
I20260927 18:39:21.543795 19041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:21.553503 19041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:21.555583 19041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:21.555755 19041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:21.560773 19041 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.252:39555
I20260927 18:39:21.560803 19108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.252:39555 every 8 connection(s)
I20260927 18:39:21.561260 19041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/data/info.pb
I20260927 18:39:21.562516 19109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:21.564955 19109 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:39:21.569178 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 19041
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534361, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.252@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.251.252@KRBTEST.COM
I20260927 18:39:21.576210 18427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.566028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54028 (local address 127.11.251.254:34227)
0927 18:39:21.566125 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.566128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.566232 (+ 104us) server_negotiation.cc:407] Connection header received
0927 18:39:21.566795 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.566797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.566812 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.566868 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.567550 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.568107 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.569222 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.569480 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.569673 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.572593 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.572595 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.572595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.572604 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.574779 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.575188 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.575189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.575189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.575258 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.575510 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.575512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.575512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.575678 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.575741 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.576017 (+ 276us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.576102 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:39:21.576637 19111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.565871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.0.0.1:54028)
0927 18:39:21.566082 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.566134 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.566681 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.566940 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.566944 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.567233 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:21.567448 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.567453 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.568214 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.568215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.569120 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.569124 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.569693 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.569694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.569885 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.570549 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:21.570557 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:21.572452 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:21.574858 (+ 2406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.574864 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.574865 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.575079 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.575324 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.575325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.575326 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.575430 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.575738 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:21.575745 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:21.575874 (+ 129us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.575980 (+ 106us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.576174 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
I20260927 18:39:21.577425 19109 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:39:21.578883 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.569803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46164 (local address 127.11.251.252:39555)
0927 18:39:21.569976 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.569981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.569999 (+ 18us) server_negotiation.cc:407] Connection header received
0927 18:39:21.570020 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.570022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.570043 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.570143 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.570933 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.571773 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.572634 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.573091 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.573164 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.576013 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.576021 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.576021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.576028 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.577684 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.578049 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.578051 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.578052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.578114 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.578311 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.578312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.578313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.578458 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.578648 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.578694 (+ 46us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.578766 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":117,"threads_started":1}
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534361, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.252@KRBTEST.COM for oryx/127.11.251.253@KRBTEST.COM
I20260927 18:39:21.584319 19111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.577729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:41041 (local address 127.0.0.1:50352)
0927 18:39:21.577851 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.577861 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.577910 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.578045 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.578046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.578180 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:21.578226 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.578227 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.578927 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.578928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.579660 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.579661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.580060 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.580061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.580135 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.580454 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:21.580457 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:21.581770 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:21.583242 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.583243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.583243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.583435 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.583644 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.583645 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.583645 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.583701 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.583984 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:21.583985 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:21.584033 (+ 48us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.584118 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.584234 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260927 18:39:21.584360 19026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.577794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50352 (local address 127.11.251.253:41041)
0927 18:39:21.577857 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.577859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.577895 (+ 36us) server_negotiation.cc:407] Connection header received
0927 18:39:21.577973 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.577974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.577985 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.578044 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.578311 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.578831 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.579746 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.579979 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.580063 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.581863 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.581865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.581865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.581873 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.583144 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.583503 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.583505 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.583505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.583572 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.583773 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.583774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.583774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.583906 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.583974 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.584167 (+ 193us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.584261 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 18:39:21.585070 19109 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534361, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.252@KRBTEST.COM for oryx/127.11.251.252@KRBTEST.COM
I20260927 18:39:21.592197 19111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.585382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:39555 (local address 127.0.0.1:46166)
0927 18:39:21.585417 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.585428 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.585488 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.586818 (+ 1330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.586819 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.586933 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:21.586957 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.586958 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.587576 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.587577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.588022 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.588023 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.588367 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.588367 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.588403 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.588722 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:21.588726 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:21.589873 (+ 1147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:21.591224 (+ 1351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.591225 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.591226 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.591399 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.591602 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.591602 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.591603 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.591655 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.591896 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:21.591897 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:21.591932 (+ 35us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.591999 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.592091 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260927 18:39:21.592248 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.585672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46166 (local address 127.11.251.252:39555)
0927 18:39:21.586678 (+ 1006us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.586680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.586690 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:39:21.586710 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.586710 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.586724 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.586784 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.587021 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.587479 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.588102 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.588296 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.588361 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.589965 (+ 1604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.589967 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.589967 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.589974 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.591128 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.591481 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.591482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.591482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.591537 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.591724 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.591725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.591725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.591833 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.591905 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.592029 (+ 124us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.592139 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":984}
I20260927 18:39:21.594348 19109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903: Bootstrap starting.
I20260927 18:39:21.595533 19109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:21.595793 19109 log.cc:824] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:21.596550 19109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903: No bootstrap required, opened a new log
I20260927 18:39:21.598237 19109 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } }
I20260927 18:39:21.598368 19109 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:21.598392 19109 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5cb1a26dafa4b76ab1b5cadcab65903, State: Initialized, Role: FOLLOWER
I20260927 18:39:21.598518 19109 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } }
I20260927 18:39:21.598784 19114 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } } }
I20260927 18:39:21.598860 19109 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:21.598872 19114 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [sys.catalog]: This master's current role is: FOLLOWER
W20260927 18:39:21.600700 19125 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:39:21.600760 19125 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:39:21.603529 19041 master_runner.cc:428] Detected that this master f5cb1a26dafa4b76ab1b5cadcab65903 is joining an existing cluster
I20260927 18:39:21.603569 19041 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.251.252:39555
I20260927 18:39:21.612506 19133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.604725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:39555 (local address 127.0.0.1:46168)
0927 18:39:21.604927 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.604939 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.605013 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.605164 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.605165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.606341 (+ 1176us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:21.606445 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.606448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.607424 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.607425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.608395 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.608398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.608865 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.608866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.608921 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.609473 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:21.609477 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:21.609885 (+ 408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:21.611440 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.611440 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.611441 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.611613 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.611820 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.611821 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.611821 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.611876 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.612122 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:21.612123 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:21.612173 (+ 50us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.612260 (+ 87us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.612370 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":4,"thread_start_us":115,"threads_started":1}
I20260927 18:39:21.612571 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.604935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46168 (local address 127.11.251.252:39555)
0927 18:39:21.604975 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.604977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.604984 (+ 7us) server_negotiation.cc:407] Connection header received
0927 18:39:21.605075 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.605076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.605086 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.605147 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.606534 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.607325 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.608485 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.608749 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.608786 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.609998 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.610000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.610000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.610007 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.611347 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.611697 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.611698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.611698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.611751 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.611946 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.611946 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.611947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.612058 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.612133 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.612280 (+ 147us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.612444 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":25}
I20260927 18:39:21.630928 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 18450
I20260927 18:39:21.636281 18427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.604672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54030 (local address 127.11.251.254:34227)
0927 18:39:21.604735 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.604738 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.605188 (+ 450us) server_negotiation.cc:407] Connection header received
0927 18:39:21.605250 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.605252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.605263 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.605317 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.606604 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.607396 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.608538 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.608845 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.608885 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.631829 (+ 22944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.631831 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.631832 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.631843 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.634680 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.635292 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.635293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.635293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.635355 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.635604 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.635606 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.635606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.635766 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.635814 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.636077 (+ 263us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.636174 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 18:39:21.636281 19026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.604939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50364 (local address 127.11.251.253:41041)
0927 18:39:21.604996 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.604998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.605084 (+ 86us) server_negotiation.cc:407] Connection header received
0927 18:39:21.605333 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.605334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.605345 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.605377 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.606667 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.607410 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.608547 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.608860 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.608898 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.631850 (+ 22952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.631852 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.631853 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.631865 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.634686 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.635294 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.635296 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.635296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.635356 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.635612 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.635613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.635613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.635766 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.635814 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.636077 (+ 263us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.636174 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 18:39:21.636699 19131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.604731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:41041 (local address 127.0.0.1:50364)
0927 18:39:21.605023 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.605032 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.605257 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.605441 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.605442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.606532 (+ 1090us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:21.606587 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.606588 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.607526 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.607527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.608454 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.608456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.608972 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.608973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.609037 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.630438 (+ 21401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:21.630443 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:21.630985 (+ 542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:21.634788 (+ 3803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.634790 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.634790 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.635195 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.635435 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.635436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.635437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.635525 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.635865 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:21.635866 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:21.635937 (+ 71us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.636423 (+ 486us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.636590 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":76,"thread_start_us":98,"threads_started":1}
I20260927 18:39:21.637300 19132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.604804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.0.0.1:54030)
0927 18:39:21.605133 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.605142 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.605195 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.605363 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.605363 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.606420 (+ 1057us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:21.606498 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.606500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.607495 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.607496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.608438 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.608441 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.608973 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.608974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.609041 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.630290 (+ 21249us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:21.630297 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:21.631702 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:21.634788 (+ 3086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.634790 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.634790 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.635195 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.635435 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.635436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.635437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.635529 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.635865 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:21.635866 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:21.635948 (+ 82us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.637052 (+ 1104us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.637191 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":48,"thread_start_us":66,"threads_started":1}
I20260927 18:39:21.638686 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:36183
--local_ip_for_outbound_sockets=127.11.251.193
--tserver_master_addrs=127.11.251.254:34227
--webserver_port=45831
--webserver_interface=127.11.251.193
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.193.keytab
--principal=oryx/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:40723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.251.254:34227,127.11.251.253:41041
--tserver_master_addrs=127.11.251.254:34227,127.11.251.253:41041,127.11.251.252:39555 with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534360, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.254@KRBTEST.COM for oryx/127.11.251.252@KRBTEST.COM
I20260927 18:39:21.648298 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.639827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:39555 (local address 127.0.0.1:46176)
0927 18:39:21.639883 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.639898 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.639963 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.640177 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.640178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.640378 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:21.640446 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.640448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.641383 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.641384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.642314 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.642317 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.642812 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.642813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.642890 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.643338 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:21.643343 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:21.645130 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:21.647046 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.647047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.647048 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.647259 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.647513 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.647514 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.647514 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.647579 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.647922 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:21.647923 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:21.647980 (+ 57us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.648052 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.648177 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260927 18:39:21.648319 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.639865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46176 (local address 127.11.251.252:39555)
0927 18:39:21.639904 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.639906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.639948 (+ 42us) server_negotiation.cc:407] Connection header received
0927 18:39:21.640034 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.640035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.640043 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.640112 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.640536 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.641309 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.642398 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.642687 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.642756 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.645224 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.645226 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.645226 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.645233 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.646943 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.647332 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.647333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.647334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.647432 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.647680 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.647681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.647681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.647841 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.647927 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.648097 (+ 170us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.648187 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 18:39:21.649381 18387 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.251.252:39555: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903"
member_type: NON_VOTER
last_known_addr {
host: "127.11.251.252"
port: 39555
}
attrs {
promote: true
}
}
dest_uuid: "350e6f81203343e4ae0976f7e03d387e"
cas_config_opid_index: 5
I20260927 18:39:21.649745 18387 consensus_queue.cc:237] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: NON_VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: true } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: NON_VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: true } }
I20260927 18:39:21.650346 18949 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f5cb1a26dafa4b76ab1b5cadcab65903 (127.11.251.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: NON_VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: true } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: NON_VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: true } } }
I20260927 18:39:21.651722 18949 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.251.252:39555
I20260927 18:39:21.651806 18949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "350e6f81203343e4ae0976f7e03d387e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: NON_VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: true } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: NON_VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: true } } }
I20260927 18:39:21.651876 18949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:21.652290 19041 master.cc:537] Master@127.11.251.252:39555 shutting down...
W20260927 18:39:21.653048 18373 consensus_peers.cc:487] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e -> Peer f5cb1a26dafa4b76ab1b5cadcab65903 (127.11.251.252:39555): Couldn't send request to peer f5cb1a26dafa4b76ab1b5cadcab65903. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260927 18:39:21.657999 19041 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:21.658114 19041 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:21.658174 19041 tablet_replica.cc:330] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903: stopping tablet replica
I20260927 18:39:21.659394 19026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.652167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50368 (local address 127.11.251.253:41041)
0927 18:39:21.652218 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.652220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.652302 (+ 82us) server_negotiation.cc:407] Connection header received
0927 18:39:21.655238 (+ 2936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.655240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.655274 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.655322 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:21.655578 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.656413 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.657869 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.658524 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.658577 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.658652 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.659262 (+ 610us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.659316 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 18:39:21.659759 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.652084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:41041 (local address 127.0.0.1:50368)
0927 18:39:21.652249 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.652260 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.652327 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.655453 (+ 3126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.655454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.655456 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:21.655500 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.655502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.656501 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.656502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.657764 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.657766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.658811 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.658812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.659153 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.659178 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.659642 (+ 464us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.659692 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149}
I20260927 18:39:21.660230 18997 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 1 LEARNER]: Refusing update from remote peer 350e6f81203343e4ae0976f7e03d387e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260927 18:39:21.660560 18947 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: NON_VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260927 18:39:21.661185 18997 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f5cb1a26dafa4b76ab1b5cadcab65903 (127.11.251.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: NON_VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: true } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: NON_VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: true } } }
I20260927 18:39:21.662820 19027 sys_catalog.cc:455] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 350e6f81203343e4ae0976f7e03d387e. Latest consensus state: current_term: 1 leader_uuid: "350e6f81203343e4ae0976f7e03d387e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: NON_VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: true } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: NON_VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: true } } }
I20260927 18:39:21.662917 19027 sys_catalog.cc:458] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [sys.catalog]: This master's current role is: LEARNER
I20260927 18:39:21.662953 19027 sys_catalog.cc:455] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "350e6f81203343e4ae0976f7e03d387e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: NON_VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: true } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: NON_VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: true } } }
I20260927 18:39:21.662969 18947 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e: attempting to promote NON_VOTER fda725da13614957a67aa50a78c8770e to VOTER
I20260927 18:39:21.662988 19027 sys_catalog.cc:458] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [sys.catalog]: This master's current role is: LEARNER
I20260927 18:39:21.663174 18947 consensus_queue.cc:237] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: NON_VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: true } }
I20260927 18:39:21.663552 18997 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 1 LEARNER]: Refusing update from remote peer 350e6f81203343e4ae0976f7e03d387e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260927 18:39:21.663641 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.663536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:39555 (local address 127.0.0.1:46186)
0927 18:39:21.663577 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.663616 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:39555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260927 18:39:21.663717 18948 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260927 18:39:21.664062 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.663978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:39555 (local address 127.0.0.1:46188)
0927 18:39:21.664021 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.664041 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:39555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260927 18:39:21.664173 18373 consensus_peers.cc:487] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e -> Peer f5cb1a26dafa4b76ab1b5cadcab65903 (127.11.251.252:39555): Couldn't send request to peer f5cb1a26dafa4b76ab1b5cadcab65903. Status: Network error: Client connection negotiation failed: client connection to 127.11.251.252:39555: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260927 18:39:21.664295 18947 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, fda725da13614957a67aa50a78c8770e (127.11.251.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: NON_VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: true } } }
I20260927 18:39:21.664427 18997 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, fda725da13614957a67aa50a78c8770e (127.11.251.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: NON_VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: true } } }
I20260927 18:39:21.665306 18947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "350e6f81203343e4ae0976f7e03d387e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: NON_VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: true } } }
I20260927 18:39:21.665371 18947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:21.665342 18948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "350e6f81203343e4ae0976f7e03d387e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: NON_VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: true } } }
I20260927 18:39:21.665421 18948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:21.665859 19136 sys_catalog.cc:455] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "350e6f81203343e4ae0976f7e03d387e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: NON_VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: true } } }
I20260927 18:39:21.665946 19136 sys_catalog.cc:458] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:39:21.671696 19041 master.cc:552] Master@127.11.251.252:39555 shutdown complete.
I20260927 18:39:21.671747 19041 master_runner.cc:305] Clearing existing system tablet
I20260927 18:39:21.671888 19041 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:21.671936 19041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:21.672708 19041 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260927 18:39:21.673257 19138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:21.673476 19041 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:21.673535 19041 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 18:39:21.673559 19041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/wal
uuid: "f5cb1a26dafa4b76ab1b5cadcab65903"
format_stamp: "Formatted at 2026-09-27 18:39:21 on dist-test-slave-z47j"
server_key: "837b6ef2295939ea066e4648315a62a7"
server_key_iv: "7527d510ed4e28886d49442389c917d3"
server_key_version: "encryptionkey@0"
I20260927 18:39:21.674422 19041 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:21.675992 19041 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260927 18:39:21.676059 19041 log.cc:1196] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260927 18:39:21.676275 19041 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903: Deleting consensus metadata
I20260927 18:39:21.676380 19041 master_runner.cc:315] Copying system tablet from 127.11.251.254:34227
I20260927 18:39:21.677078 19041 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903: tablet copy: Beginning tablet copy session from remote peer at address 127.11.251.254:34227
I20260927 18:39:21.684782 19144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.677402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.0.0.1:54042)
0927 18:39:21.677528 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.677539 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.677635 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.677831 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.677832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.678098 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:21.678177 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.678181 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.679188 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.679189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.680080 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.680083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.680523 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.680524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.680594 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.681159 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:21.681165 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:21.681653 (+ 488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:21.683627 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.683628 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.683629 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.683847 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.684078 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.684079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.684079 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.684145 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.684441 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:21.684442 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:21.684495 (+ 53us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.684571 (+ 76us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.684680 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":70,"threads_started":1}
I20260927 18:39:21.684784 18427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.677462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54042 (local address 127.11.251.254:34227)
0927 18:39:21.677509 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.677512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.677617 (+ 105us) server_negotiation.cc:407] Connection header received
0927 18:39:21.677730 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.677731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.677743 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.677815 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.678273 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.679096 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.680174 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.680438 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.680511 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.681739 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.681741 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.681741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.681749 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.683528 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.683929 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.683930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.683931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.684004 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.684219 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.684220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.684220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.684367 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.684444 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.684615 (+ 171us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.684695 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 18:39:21.685492 18408 tablet_copy_service.cc:140] P 350e6f81203343e4ae0976f7e03d387e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f5cb1a26dafa4b76ab1b5cadcab65903 ({username='oryx', principal='oryx/127.11.251.252@KRBTEST.COM'} at 127.0.0.1:54042)
I20260927 18:39:21.685557 18408 tablet_copy_service.cc:161] P 350e6f81203343e4ae0976f7e03d387e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f5cb1a26dafa4b76ab1b5cadcab65903 at {username='oryx', principal='oryx/127.11.251.252@KRBTEST.COM'} at 127.0.0.1:54042: session id = f5cb1a26dafa4b76ab1b5cadcab65903-00000000000000000000000000000000
I20260927 18:39:21.686313 18408 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e: Tablet Copy: opened 0 blocks and 1 log segments
I20260927 18:39:21.686859 19041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260927 18:39:21.688230 19041 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903: tablet copy: Starting download of 0 data blocks...
I20260927 18:39:21.688380 19041 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903: tablet copy: Starting download of 1 WAL segments...
I20260927 18:39:21.689232 19041 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260927 18:39:21.690464 18408 tablet_copy_service.cc:342] P 350e6f81203343e4ae0976f7e03d387e: Request end of tablet copy session f5cb1a26dafa4b76ab1b5cadcab65903-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.251.252@KRBTEST.COM'} at 127.0.0.1:54042
I20260927 18:39:21.690528 18408 tablet_copy_service.cc:433] P 350e6f81203343e4ae0976f7e03d387e: ending tablet copy session f5cb1a26dafa4b76ab1b5cadcab65903-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f5cb1a26dafa4b76ab1b5cadcab65903
I20260927 18:39:21.694177 19041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:21.695448 19145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:21.695495 19148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:21.695801 19146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:21.698100 19041 server_base.cc:1034] running on GCE node
I20260927 18:39:21.698215 19041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:21.698380 19041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:21.699529 19041 hybrid_clock.cc:648] HybridClock initialized: now 1790534361699512 us; error 43 us; skew 500 ppm
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534361, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:21.701202 19041 init.cc:377] Logged in from keytab as oryx/127.11.251.252@KRBTEST.COM (short username oryx)
I20260927 18:39:21.701489 19041 webserver.cc:460] Webserver started at http://127.11.251.252:41885/ using document root <none> and password file <none>
I20260927 18:39:21.701663 19041 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:21.701709 19041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:21.702507 19041 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:39:21.703127 19155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:21.703272 19041 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:39:21.703346 19041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/wal
uuid: "f5cb1a26dafa4b76ab1b5cadcab65903"
format_stamp: "Formatted at 2026-09-27 18:39:21 on dist-test-slave-z47j"
server_key: "837b6ef2295939ea066e4648315a62a7"
server_key_iv: "7527d510ed4e28886d49442389c917d3"
server_key_version: "encryptionkey@0"
I20260927 18:39:21.703415 19041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:21.731284 19041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:21.734372 19041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:21.738353 19041 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.252:39555
I20260927 18:39:21.738461 19207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.252:39555 every 8 connection(s)
I20260927 18:39:21.738741 19041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/master-2/data/info.pb
I20260927 18:39:21.742501 19208 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:39:21.743193 19208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903: Bootstrap starting.
I20260927 18:39:21.746817 19208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:39:21.747053 19208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903: Bootstrap complete.
I20260927 18:39:21.747319 19208 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: NON_VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: true } }
I20260927 18:39:21.747429 19208 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f5cb1a26dafa4b76ab1b5cadcab65903, State: Initialized, Role: LEARNER
I20260927 18:39:21.747468 19208 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: NON_VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: true } }
I20260927 18:39:21.747689 19208 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:21.747702 19211 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: NON_VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: true } } }
I20260927 18:39:21.747933 19211 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [sys.catalog]: This master's current role is: LEARNER
I20260927 18:39:21.749410 19222 catalog_manager.cc:1259] Loaded cluster ID: 08175ddff41c4f508594ea58d66b74dc
I20260927 18:39:21.749497 19222 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903: loading cluster ID for follower catalog manager: success
I20260927 18:39:21.750926 19222 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903: acquiring CA information for follower catalog manager: success
I20260927 18:39:21.751242 19222 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260927 18:39:21.757990 19134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:21.758229 19134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:21.758281 19134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:21.759542 19134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:21.759613 19134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:21.759658 19134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:39:21.761765 19134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:40723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.193.keytab
--principal=oryx/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:36183
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=45831
--tserver_master_addrs=127.11.251.254:34227,127.11.251.253:41041,127.11.251.252:39555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:21.762050 19134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:21.762315 19134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:21.767722 19228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:21.767721 19230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:21.767741 19227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:21.768342 19134 server_base.cc:1034] running on GCE node
I20260927 18:39:21.768556 19134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:21.768836 19134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:21.770035 19134 hybrid_clock.cc:648] HybridClock initialized: now 1790534361770017 us; error 46 us; skew 500 ppm
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534361, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:21.773165 19134 init.cc:377] Logged in from keytab as oryx/127.11.251.193@KRBTEST.COM (short username oryx)
I20260927 18:39:21.773742 19134 webserver.cc:460] Webserver started at http://127.11.251.193:45831/ using document root <none> and password file <none>
I20260927 18:39:21.773993 19134 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:21.774080 19134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:21.775691 19134 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:39:21.776353 19237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:21.776530 19134 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:21.776662 19134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "159be8c77cdc42d1b1a0840730dd6737"
format_stamp: "Formatted at 2026-09-27 18:39:20 on dist-test-slave-z47j"
server_key: "869d3d3daa8e26395de12eccc34da480"
server_key_iv: "b68ad690d55e6b32a6cd3c8eb57ebf0d"
server_key_version: "encryptionkey@0"
I20260927 18:39:21.777060 19134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:21.791774 19134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:21.794445 19134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:21.794632 19134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:21.794896 19134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:21.795282 19134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:21.795336 19134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:21.795379 19134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:21.795409 19134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:21.802474 19134 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:36183
I20260927 18:39:21.802624 19350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:36183 every 8 connection(s)
I20260927 18:39:21.802949 19134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:39:21.805369 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 19134
I20260927 18:39:21.805606 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 18584
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534361, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.252@KRBTEST.COM
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534361, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.253@KRBTEST.COM
I20260927 18:39:21.815147 19026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.803895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:39765 (local address 127.11.251.253:41041)
0927 18:39:21.803960 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.803962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.804431 (+ 469us) server_negotiation.cc:407] Connection header received
0927 18:39:21.805458 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.805461 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.805481 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.805556 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.806529 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.807354 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.809266 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.809528 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.809648 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.811907 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.811909 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.811909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.811918 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.813666 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.814084 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.814086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.814086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.814146 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.814351 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.814351 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.814352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.814504 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.814547 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.814900 (+ 353us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.814995 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 18:39:21.815455 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:43837
--local_ip_for_outbound_sockets=127.11.251.194
--tserver_master_addrs=127.11.251.254:34227
--webserver_port=36161
--webserver_interface=127.11.251.194
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.194.keytab
--principal=oryx/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:40723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.251.254:34227,127.11.251.253:41041
--tserver_master_addrs=127.11.251.254:34227,127.11.251.253:41041,127.11.251.252:39555 with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:39:21.815500 19357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.803872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:44217 (local address 127.11.251.252:39555)
0927 18:39:21.805707 (+ 1835us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.805709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.805718 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:39:21.805750 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.805751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.805764 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.806534 (+ 770us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.806594 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.807293 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.808362 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.808605 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.808697 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.811663 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.811665 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.811665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.811673 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.813417 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.813969 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.813971 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.813971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.814045 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.814312 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.814313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.814313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.814472 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.814537 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.815311 (+ 774us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.815402 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1810,"thread_start_us":72,"threads_started":1}
I20260927 18:39:21.815568 19358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.803839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:41041 (local address 127.11.251.193:39765)
0927 18:39:21.804360 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.804371 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.805323 (+ 952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.805680 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.805684 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.806193 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:21.806424 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.806429 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.807453 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.807454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.809173 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.809176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.809634 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.809635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.809715 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.810146 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:21.810151 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:21.811810 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:21.813751 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.813752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.813753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.814000 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.814227 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.814228 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.814228 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.814286 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.814602 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:21.814602 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:21.814747 (+ 145us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.815253 (+ 506us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.815371 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":2,"thread_start_us":100,"threads_started":1}
I20260927 18:39:21.815611 19356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.803820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:39555 (local address 127.11.251.193:44217)
0927 18:39:21.804100 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.804134 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.805323 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.805855 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.805855 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.806109 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:21.806424 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.806429 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.807414 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.807415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.808265 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.808267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.808702 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.808703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.808845 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.809741 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:21.809750 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:21.811547 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:21.813500 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.813506 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.813509 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.813888 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.814112 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.814113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.814113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.814243 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.814562 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:21.814569 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:21.814747 (+ 178us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.814854 (+ 107us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.815034 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":6,"thread_start_us":107,"threads_started":1}
I20260927 18:39:21.816452 19353 heartbeater.cc:347] Connected to a master server at 127.11.251.253:41041
I20260927 18:39:21.816579 19353 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:21.816810 19353 heartbeater.cc:511] Master 127.11.251.253:41041 requested a full tablet report, sending...
I20260927 18:39:21.817468 18987 ts_manager.cc:194] Registered new tserver with Master: 159be8c77cdc42d1b1a0840730dd6737 (127.11.251.193:36183)
I20260927 18:39:21.817862 19351 heartbeater.cc:347] Connected to a master server at 127.11.251.252:39555
I20260927 18:39:21.817920 19351 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:21.818013 19351 heartbeater.cc:511] Master 127.11.251.252:39555 requested a full tablet report, sending...
I20260927 18:39:21.818567 19172 ts_manager.cc:194] Registered new tserver with Master: 159be8c77cdc42d1b1a0840730dd6737 (127.11.251.193:36183)
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534361, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM
I20260927 18:39:21.822188 19355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.804168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.193:38307)
0927 18:39:21.805499 (+ 1331us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.805519 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.805609 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.814386 (+ 8777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.814388 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.814581 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:21.814649 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.814651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.816112 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.816113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.816894 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.816896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.817334 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.817335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.817403 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.817829 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:21.817834 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:21.819383 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:21.821136 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.821137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.821138 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.821319 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.821551 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.821551 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.821552 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.821623 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.821862 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:21.821863 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:21.821905 (+ 42us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.821966 (+ 61us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.822081 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1316,"thread_start_us":91,"threads_started":1}
I20260927 18:39:21.822191 18427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.804281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:38307 (local address 127.11.251.254:34227)
0927 18:39:21.804355 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.804357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.814223 (+ 9866us) server_negotiation.cc:407] Connection header received
0927 18:39:21.814261 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.814263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.814278 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.814332 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.814747 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.815483 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.816963 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.817247 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.817326 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.819476 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.819478 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.819478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.819486 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.821044 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.821405 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.821407 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.821407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.821471 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.821671 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.821673 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.821673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.821793 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.821875 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.822022 (+ 147us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.822097 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 18:39:21.822700 19352 heartbeater.cc:347] Connected to a master server at 127.11.251.254:34227
I20260927 18:39:21.822738 19352 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:21.822789 19352 heartbeater.cc:511] Master 127.11.251.254:34227 requested a full tablet report, sending...
I20260927 18:39:21.823100 18387 ts_manager.cc:194] Re-registered known tserver with Master: 159be8c77cdc42d1b1a0840730dd6737 (127.11.251.193:36183)
I20260927 18:39:21.823510 18387 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:38307
W20260927 18:39:21.910000 19359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:21.910310 19359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:21.910349 19359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:21.911614 19359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:21.911682 19359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:21.911731 19359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:39:21.913337 19359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:40723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.194.keytab
--principal=oryx/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:43837
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=36161
--tserver_master_addrs=127.11.251.254:34227,127.11.251.253:41041,127.11.251.252:39555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:21.913699 19359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:21.913980 19359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:21.919953 19364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:21.919958 19367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:21.919924 19365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:21.919993 19359 server_base.cc:1034] running on GCE node
I20260927 18:39:21.920383 19359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:21.920660 19359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:21.921849 19359 hybrid_clock.cc:648] HybridClock initialized: now 1790534361921831 us; error 39 us; skew 500 ppm
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534361, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:21.924747 19359 init.cc:377] Logged in from keytab as oryx/127.11.251.194@KRBTEST.COM (short username oryx)
I20260927 18:39:21.925213 19359 webserver.cc:460] Webserver started at http://127.11.251.194:36161/ using document root <none> and password file <none>
I20260927 18:39:21.925442 19359 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:21.925505 19359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:21.927047 19359 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:39:21.927655 19374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:21.927865 19359 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:39:21.927977 19359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "310dc769e00b4c279134a9d9b0f8bfe9"
format_stamp: "Formatted at 2026-09-27 18:39:20 on dist-test-slave-z47j"
server_key: "96ea273a53e7b80526ce96cf76e1b1ff"
server_key_iv: "e5d9eb19dce19aeb8667fb6a4f4144ae"
server_key_version: "encryptionkey@0"
I20260927 18:39:21.928386 19359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:21.941053 19359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:21.943279 19359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:21.943447 19359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:21.943749 19359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:21.944114 19359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:21.944149 19359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:21.944175 19359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:21.944221 19359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:21.951442 19359 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:43837
I20260927 18:39:21.951529 19487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:43837 every 8 connection(s)
I20260927 18:39:21.951996 19359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:39:21.952679 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 19359
I20260927 18:39:21.952895 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 18718
I20260927 18:39:21.961308 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:37335
--local_ip_for_outbound_sockets=127.11.251.195
--tserver_master_addrs=127.11.251.254:34227
--webserver_port=35595
--webserver_interface=127.11.251.195
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.195.keytab
--principal=oryx/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:40723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.251.254:34227,127.11.251.253:41041
--tserver_master_addrs=127.11.251.254:34227,127.11.251.253:41041,127.11.251.252:39555 with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534361, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.252@KRBTEST.COM
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534361, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM
Sep 27 18:39:21 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534361, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.253@KRBTEST.COM
I20260927 18:39:21.967779 19357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.955072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:40279 (local address 127.11.251.252:39555)
0927 18:39:21.955130 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.955132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.955452 (+ 320us) server_negotiation.cc:407] Connection header received
0927 18:39:21.955562 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.955563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.955581 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.955613 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.956002 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.956787 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.958623 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.958945 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.958986 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.963899 (+ 4913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.963901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.963901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.963911 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.966152 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.966583 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.966584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.966585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.966648 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.966889 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.966889 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.966890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.967054 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.967099 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.967563 (+ 464us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.967674 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 18:39:21.967795 19026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.953230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:35741 (local address 127.11.251.253:41041)
0927 18:39:21.954271 (+ 1041us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.954273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.954288 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:39:21.954624 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.954626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.954643 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.954718 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.955870 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.956793 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.958568 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.958877 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.958921 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.963779 (+ 4858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.963781 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.963782 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.963792 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.965983 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.966521 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.966523 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.966523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.966601 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.966893 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.966894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.966895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.967042 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.967099 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.967570 (+ 471us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.967670 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":996}
I20260927 18:39:21.967846 18427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.953016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:59343 (local address 127.11.251.254:34227)
0927 18:39:21.953512 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:21.953514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:21.954808 (+ 1294us) server_negotiation.cc:407] Connection header received
0927 18:39:21.954911 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:21.954913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:21.954928 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:21.954962 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:21.955896 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.956754 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.958579 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.958941 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.958989 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.963758 (+ 4769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:21.963760 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:21.963761 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:21.963771 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:21.965847 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.966617 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.966619 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.966619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.966687 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:21.966901 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:21.966902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:21.966903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:21.967077 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:21.967136 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:21.967588 (+ 452us) server_negotiation.cc:299] Negotiation successful
0927 18:39:21.967697 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453}
I20260927 18:39:21.968556 19492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.952890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.194:59343)
0927 18:39:21.954735 (+ 1845us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.954747 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.954836 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.955055 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.955056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.955426 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:21.955759 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.955767 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.956909 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.956910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.958461 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.958464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.959080 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.959081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.959192 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.960225 (+ 1033us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:21.960230 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:21.963625 (+ 3395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:21.965942 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.965950 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.965953 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.966528 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.966753 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.966754 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.966754 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.966834 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.967171 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:21.967171 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:21.967370 (+ 199us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.967993 (+ 623us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.968316 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1814,"mutex_wait_us":392,"thread_start_us":95,"threads_started":1}
I20260927 18:39:21.968693 19493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.952890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:41041 (local address 127.11.251.194:35741)
0927 18:39:21.953536 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.953617 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.954489 (+ 872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.954878 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.954882 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.955389 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:21.955759 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.955765 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.956919 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.956920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.958462 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.958464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.959032 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.959033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.959192 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.960053 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:21.960067 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:21.963625 (+ 3558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:21.966090 (+ 2465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.966091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.966092 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.966423 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.966676 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.966677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.966677 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.966815 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.967134 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:21.967142 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:21.967370 (+ 228us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.967936 (+ 566us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.968216 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":83,"thread_start_us":106,"threads_started":1}
I20260927 18:39:21.969134 19489 heartbeater.cc:347] Connected to a master server at 127.11.251.254:34227
I20260927 18:39:21.969257 19489 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:21.969300 19490 heartbeater.cc:347] Connected to a master server at 127.11.251.253:41041
I20260927 18:39:21.969363 19490 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:21.969513 19490 heartbeater.cc:511] Master 127.11.251.253:41041 requested a full tablet report, sending...
I20260927 18:39:21.969527 19489 heartbeater.cc:511] Master 127.11.251.254:34227 requested a full tablet report, sending...
I20260927 18:39:21.969709 19494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:21.954844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:39555 (local address 127.11.251.194:40279)
0927 18:39:21.955386 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:21.955396 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:21.955477 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:21.955684 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:21.955684 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:21.955860 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:21.955922 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.955923 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.956919 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.956920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.958538 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:21.958540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:21.959061 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:21.959062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:21.959192 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:21.960086 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:21.960090 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:21.963770 (+ 3680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:21.966238 (+ 2468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.966239 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.966240 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.966498 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.966725 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:21.966726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:21.966726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:21.966814 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:21.967147 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:21.967148 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:21.967370 (+ 222us) client_negotiation.cc:769] Sending connection context
0927 18:39:21.968486 (+ 1116us) client_negotiation.cc:240] Negotiation successful
0927 18:39:21.969447 (+ 961us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"mutex_wait_us":694,"thread_start_us":96,"threads_started":1}
I20260927 18:39:21.970054 18387 ts_manager.cc:194] Re-registered known tserver with Master: 310dc769e00b4c279134a9d9b0f8bfe9 (127.11.251.194:43837)
I20260927 18:39:21.970134 18987 ts_manager.cc:194] Registered new tserver with Master: 310dc769e00b4c279134a9d9b0f8bfe9 (127.11.251.194:43837)
I20260927 18:39:21.970728 18387 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:59343
I20260927 18:39:21.970891 19488 heartbeater.cc:347] Connected to a master server at 127.11.251.252:39555
I20260927 18:39:21.970935 19488 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:21.971010 19488 heartbeater.cc:511] Master 127.11.251.252:39555 requested a full tablet report, sending...
I20260927 18:39:21.971572 19172 ts_manager.cc:194] Registered new tserver with Master: 310dc769e00b4c279134a9d9b0f8bfe9 (127.11.251.194:43837)
W20260927 18:39:22.060042 19495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:22.060271 19495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:22.060294 19495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:22.061424 19495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:22.061465 19495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:22.061492 19495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:39:22.062933 19495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:40723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.195.keytab
--principal=oryx/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:37335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=35595
--tserver_master_addrs=127.11.251.254:34227,127.11.251.253:41041,127.11.251.252:39555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:22.063136 19495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:22.063337 19495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:22.068699 19500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:22.068666 19501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:22.068742 19495 server_base.cc:1034] running on GCE node
W20260927 18:39:22.068672 19503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:22.069087 19495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:22.069334 19495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:22.070492 19495 hybrid_clock.cc:648] HybridClock initialized: now 1790534362070466 us; error 65 us; skew 500 ppm
Sep 27 18:39:22 dist-test-slave-z47j krb5kdc[18343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534362, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:22.073320 19495 init.cc:377] Logged in from keytab as oryx/127.11.251.195@KRBTEST.COM (short username oryx)
I20260927 18:39:22.073827 19495 webserver.cc:460] Webserver started at http://127.11.251.195:35595/ using document root <none> and password file <none>
I20260927 18:39:22.074067 19495 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:22.074127 19495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:22.075466 19495 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:22.076017 19510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:22.076231 19495 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:22.076339 19495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "be295e1ae1234a83bb005c34e6a7ad4e"
format_stamp: "Formatted at 2026-09-27 18:39:21 on dist-test-slave-z47j"
server_key: "1af185f8d75702abffdf034cd8f080e5"
server_key_iv: "07092d9a14b2de3020738646f53122d9"
server_key_version: "encryptionkey@0"
I20260927 18:39:22.076731 19495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:22.091440 19495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:22.093817 19495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:22.093978 19495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:22.094224 19495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:22.094563 19495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:22.094600 19495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:22.094624 19495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:22.094637 19495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:22.101253 19495 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:37335
I20260927 18:39:22.101337 19623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:37335 every 8 connection(s)
I20260927 18:39:22.101742 19495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:39:22.109570 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 19495
Sep 27 18:39:22 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534362, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.253@KRBTEST.COM
Sep 27 18:39:22 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534362, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM
Sep 27 18:39:22 dist-test-slave-z47j krb5kdc[18343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534362, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.252@KRBTEST.COM
I20260927 18:39:22.116425 19026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.102965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:43529 (local address 127.11.251.253:41041)
0927 18:39:22.103046 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:22.103049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:22.103270 (+ 221us) server_negotiation.cc:407] Connection header received
0927 18:39:22.104327 (+ 1057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:22.104329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:22.104343 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:22.104413 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:22.105457 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.106295 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.107607 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.107935 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.108014 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.113079 (+ 5065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:22.113081 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:22.113081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:22.113096 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:22.114824 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:22.115311 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:22.115313 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:22.115313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:22.115377 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:22.115660 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:22.115661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:22.115662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:22.115781 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:22.115876 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:22.116239 (+ 363us) server_negotiation.cc:299] Negotiation successful
0927 18:39:22.116323 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 18:39:22.116660 19630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.102587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:41041 (local address 127.11.251.195:43529)
0927 18:39:22.103191 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:22.103200 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:22.104212 (+ 1012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:22.104503 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:22.104509 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:22.105086 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:22.105346 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:22.105351 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.106377 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.106377 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:22.107498 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:22.107500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.108416 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.108417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:22.108464 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.108937 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:22.108942 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:22.112342 (+ 3400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:22.114920 (+ 2578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:22.114927 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:22.114929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:22.115229 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:22.115446 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:22.115447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:22.115448 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:22.115591 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:22.115901 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:22.115908 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:22.116092 (+ 184us) client_negotiation.cc:769] Sending connection context
0927 18:39:22.116219 (+ 127us) client_negotiation.cc:240] Negotiation successful
0927 18:39:22.116388 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"mutex_wait_us":34,"thread_start_us":105,"threads_started":1}
I20260927 18:39:22.117869 19626 heartbeater.cc:347] Connected to a master server at 127.11.251.253:41041
I20260927 18:39:22.118005 19626 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:22.118232 19626 heartbeater.cc:511] Master 127.11.251.253:41041 requested a full tablet report, sending...
I20260927 18:39:22.119117 18987 ts_manager.cc:194] Registered new tserver with Master: be295e1ae1234a83bb005c34e6a7ad4e (127.11.251.195:37335)
I20260927 18:39:22.119297 19628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.102587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.195:34907)
0927 18:39:22.102824 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:22.102872 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:22.104201 (+ 1329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:22.104852 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:22.104852 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:22.105086 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:22.105347 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:22.105353 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.107599 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.107599 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:22.108211 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:22.108213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.108658 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.108659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:22.108748 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.109096 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:22.109101 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:22.112675 (+ 3574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:22.118269 (+ 5594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:22.118270 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:22.118271 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:22.118476 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:22.118647 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:22.118648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:22.118648 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:22.118707 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:22.118988 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:22.118989 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:22.119030 (+ 41us) client_negotiation.cc:769] Sending connection context
0927 18:39:22.119091 (+ 61us) client_negotiation.cc:240] Negotiation successful
0927 18:39:22.119188 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":13,"thread_start_us":125,"threads_started":1}
I20260927 18:39:22.119895 18427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.103027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:34907 (local address 127.11.251.254:34227)
0927 18:39:22.103088 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:22.103091 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:22.103103 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:39:22.104327 (+ 1224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:22.104329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:22.104343 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:22.104414 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:22.105492 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.106298 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.108300 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.108557 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.108627 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.112750 (+ 4123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:22.112752 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:22.112752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:22.112760 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:22.118177 (+ 5417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:22.118535 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:22.118536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:22.118536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:22.118596 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:22.118777 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:22.118778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:22.118778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:22.118905 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:22.118963 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:22.119716 (+ 753us) server_negotiation.cc:299] Negotiation successful
0927 18:39:22.119798 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 18:39:22.120214 19625 heartbeater.cc:347] Connected to a master server at 127.11.251.254:34227
I20260927 18:39:22.120280 19625 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:22.120361 19625 heartbeater.cc:511] Master 127.11.251.254:34227 requested a full tablet report, sending...
I20260927 18:39:22.120451 19357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.102575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:58241 (local address 127.11.251.252:39555)
0927 18:39:22.102633 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:22.102636 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:22.104444 (+ 1808us) server_negotiation.cc:407] Connection header received
0927 18:39:22.104547 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:22.104548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:22.104563 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:22.104613 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:22.105460 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.106075 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.107609 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.107930 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.108013 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.113228 (+ 5215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:22.113229 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:22.113230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:22.113238 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:22.118383 (+ 5145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:22.119481 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:22.119483 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:22.119483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:22.119546 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:22.119745 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:22.119746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:22.119746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:22.119879 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:22.119953 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:22.120273 (+ 320us) server_negotiation.cc:299] Negotiation successful
0927 18:39:22.120353 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 18:39:22.120611 19629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.102587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:39555 (local address 127.11.251.195:58241)
0927 18:39:22.104378 (+ 1791us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:22.104389 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:22.104469 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:22.104632 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:22.104632 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:22.105087 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:22.105347 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:22.105353 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.106183 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.106184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:22.107498 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:22.107500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.108025 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.108026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:22.108166 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.108813 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:22.108824 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:22.112985 (+ 4161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:22.118471 (+ 5486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:22.118472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:22.118473 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:22.118711 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:22.119619 (+ 908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:22.119620 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:22.119621 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:22.119676 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:22.119981 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:22.119982 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:22.120023 (+ 41us) client_negotiation.cc:769] Sending connection context
0927 18:39:22.120086 (+ 63us) client_negotiation.cc:240] Negotiation successful
0927 18:39:22.120194 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1760,"mutex_wait_us":510,"thread_start_us":85,"threads_started":1}
I20260927 18:39:22.120643 19638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.111075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50380 (local address 127.11.251.253:41041)
0927 18:39:22.112331 (+ 1256us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:22.112333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:22.112341 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:39:22.112362 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:22.112363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:22.112375 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:22.113080 (+ 705us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:22.113117 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.113897 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.114941 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.115254 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.115576 (+ 322us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.116801 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:22.116802 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:22.116802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:22.116807 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:22.119134 (+ 2327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:22.119539 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:22.119540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:22.119540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:22.119594 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:22.119810 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:22.119811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:22.119812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:22.119936 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:22.120346 (+ 410us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:22.120385 (+ 39us) server_negotiation.cc:299] Negotiation successful
0927 18:39:22.120519 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1232,"thread_start_us":85,"threads_started":1}
I20260927 18:39:22.121217 18387 ts_manager.cc:194] Re-registered known tserver with Master: be295e1ae1234a83bb005c34e6a7ad4e (127.11.251.195:37335)
I20260927 18:39:22.121229 19624 heartbeater.cc:347] Connected to a master server at 127.11.251.252:39555
I20260927 18:39:22.121279 19624 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:22.121335 19624 heartbeater.cc:511] Master 127.11.251.252:39555 requested a full tablet report, sending...
I20260927 18:39:22.121704 19172 ts_manager.cc:194] Registered new tserver with Master: be295e1ae1234a83bb005c34e6a7ad4e (127.11.251.195:37335)
I20260927 18:39:22.121726 18387 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:34907
I20260927 18:39:22.123601 19639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.111200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46190 (local address 127.11.251.252:39555)
0927 18:39:22.112141 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:22.112143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:22.112150 (+ 7us) server_negotiation.cc:407] Connection header received
0927 18:39:22.112171 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:22.112172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:22.112184 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:22.112252 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:22.112926 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.113744 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.114920 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.115250 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.115297 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.117963 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:22.117965 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:22.117965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:22.118009 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:22.122409 (+ 4400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:22.122762 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:22.122763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:22.122763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:22.122818 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:22.123032 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:22.123033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:22.123033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:22.123177 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:22.123285 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:22.123423 (+ 138us) server_negotiation.cc:299] Negotiation successful
0927 18:39:22.123487 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":920,"thread_start_us":85,"threads_started":1}
I20260927 18:39:22.125650 19636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.110676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54050 (local address 127.11.251.254:34227)
0927 18:39:22.110990 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:22.110992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:22.111169 (+ 177us) server_negotiation.cc:407] Connection header received
0927 18:39:22.111192 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:22.111193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:22.111207 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:22.111274 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:22.112097 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.118460 (+ 6363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.119619 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.120198 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.121959 (+ 1761us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.122853 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:22.122854 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:22.122855 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:22.122861 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:22.124509 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:22.124861 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:22.124862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:22.124862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:22.124923 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:22.125126 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:22.125127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:22.125127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:22.125238 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:22.125349 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:22.125436 (+ 87us) server_negotiation.cc:299] Negotiation successful
0927 18:39:22.125522 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":92,"threads_started":1}
I20260927 18:39:22.127291 18387 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:39:22.129165 18387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:39:22.137704 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.133802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:36183 (local address 127.0.0.1:34110)
0927 18:39:22.133967 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:22.133979 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:22.134039 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:22.134393 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:22.134394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:22.134397 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:22.134445 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:22.134447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.135566 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.135568 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:22.136503 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:22.136505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.137452 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.137453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:22.137512 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.137516 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:39:22.137558 (+ 42us) client_negotiation.cc:240] Negotiation successful
0927 18:39:22.137630 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":116,"threads_started":1}
I20260927 18:39:22.137990 19647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.133958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34110 (local address 127.11.251.193:36183)
0927 18:39:22.134181 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:22.134186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:22.134198 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:39:22.134248 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:22.134250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:22.134264 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:22.134395 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:22.134524 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.135481 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.136598 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.137365 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.137477 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.137558 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:22.137640 (+ 82us) server_negotiation.cc:299] Negotiation successful
0927 18:39:22.137689 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":114,"threads_started":1}
I20260927 18:39:22.138237 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.133567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:43837 (local address 127.0.0.1:40668)
0927 18:39:22.133638 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:22.133650 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:22.133692 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:22.134855 (+ 1163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:22.134856 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:22.134858 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:22.134893 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:22.134894 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.136193 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.136194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:22.137045 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:22.137047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.138008 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.138009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:22.138072 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.138076 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:39:22.138120 (+ 44us) client_negotiation.cc:240] Negotiation successful
0927 18:39:22.138167 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260927 18:39:22.138378 19645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.133702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:37335 (local address 127.0.0.1:55894)
0927 18:39:22.133873 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:22.133886 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:22.133935 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:22.134374 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:22.134375 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:22.134378 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:22.134438 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:22.134440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.136134 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.136135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:22.137224 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:22.137227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.138161 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.138162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:22.138214 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.138218 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:39:22.138246 (+ 28us) client_negotiation.cc:240] Negotiation successful
0927 18:39:22.138296 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":44,"thread_start_us":111,"threads_started":1}
I20260927 18:39:22.138705 19648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.133988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55894 (local address 127.11.251.195:37335)
0927 18:39:22.134187 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:22.134191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:22.134201 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:39:22.134246 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:22.134248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:22.134266 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:22.134513 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:22.134576 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.136045 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.137310 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.138073 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.138485 (+ 412us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.138546 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:22.138571 (+ 25us) server_negotiation.cc:299] Negotiation successful
0927 18:39:22.138624 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":128,"threads_started":1}
I20260927 18:39:22.138763 19649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.134223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40668 (local address 127.11.251.194:43837)
0927 18:39:22.134653 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:22.134657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:22.134667 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:39:22.134734 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:22.134736 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:22.134753 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:22.134844 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:22.134956 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.136086 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.137141 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.137908 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.138409 (+ 501us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.138601 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:22.138625 (+ 24us) server_negotiation.cc:299] Negotiation successful
0927 18:39:22.138668 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":90,"threads_started":1}
I20260927 18:39:22.138827 19285 tablet_service.cc:1467] Processing CreateTablet for tablet 38128a6eeaa34c41a26e77dff92fd714 (DEFAULT_TABLE table=test-table [id=1c898071255743f3897ba8ff98e8c3e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:22.139338 19285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38128a6eeaa34c41a26e77dff92fd714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:22.139350 19558 tablet_service.cc:1467] Processing CreateTablet for tablet 38128a6eeaa34c41a26e77dff92fd714 (DEFAULT_TABLE table=test-table [id=1c898071255743f3897ba8ff98e8c3e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:22.139676 19558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38128a6eeaa34c41a26e77dff92fd714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:22.142320 19422 tablet_service.cc:1467] Processing CreateTablet for tablet 38128a6eeaa34c41a26e77dff92fd714 (DEFAULT_TABLE table=test-table [id=1c898071255743f3897ba8ff98e8c3e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:22.142689 19422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38128a6eeaa34c41a26e77dff92fd714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:22.142937 19651 tablet_bootstrap.cc:492] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e: Bootstrap starting.
I20260927 18:39:22.143566 19651 tablet_bootstrap.cc:654] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:22.144436 19651 log.cc:824] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:22.144819 19650 tablet_bootstrap.cc:492] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737: Bootstrap starting.
I20260927 18:39:22.145445 19651 tablet_bootstrap.cc:492] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e: No bootstrap required, opened a new log
I20260927 18:39:22.145555 19650 tablet_bootstrap.cc:654] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:22.145617 19651 ts_tablet_manager.cc:1397] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:39:22.145895 19650 log.cc:824] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:22.146620 19650 tablet_bootstrap.cc:492] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737: No bootstrap required, opened a new log
I20260927 18:39:22.146699 19650 ts_tablet_manager.cc:1397] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260927 18:39:22.147173 19654 tablet_bootstrap.cc:492] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9: Bootstrap starting.
I20260927 18:39:22.147840 19654 tablet_bootstrap.cc:654] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:22.147730 19651 raft_consensus.cc:348] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43837 } } peers { permanent_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 37335 } } peers { permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 } }
I20260927 18:39:22.147966 19651 raft_consensus.cc:374] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:22.148007 19651 raft_consensus.cc:729] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be295e1ae1234a83bb005c34e6a7ad4e, State: Initialized, Role: FOLLOWER
I20260927 18:39:22.148133 19651 consensus_queue.cc:260] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43837 } } peers { permanent_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 37335 } } peers { permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 } }
I20260927 18:39:22.148185 19654 log.cc:824] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:22.148427 19651 ts_tablet_manager.cc:1428] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:39:22.148828 19625 heartbeater.cc:503] Master 127.11.251.254:34227 was elected leader, sending a full tablet report...
I20260927 18:39:22.149226 19654 tablet_bootstrap.cc:492] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9: No bootstrap required, opened a new log
I20260927 18:39:22.149341 19654 ts_tablet_manager.cc:1397] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:39:22.151278 19650 raft_consensus.cc:348] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43837 } } peers { permanent_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 37335 } } peers { permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 } }
I20260927 18:39:22.151439 19650 raft_consensus.cc:374] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:22.151475 19650 raft_consensus.cc:729] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 159be8c77cdc42d1b1a0840730dd6737, State: Initialized, Role: FOLLOWER
I20260927 18:39:22.151608 19650 consensus_queue.cc:260] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43837 } } peers { permanent_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 37335 } } peers { permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 } }
I20260927 18:39:22.151918 19352 heartbeater.cc:503] Master 127.11.251.254:34227 was elected leader, sending a full tablet report...
I20260927 18:39:22.151865 19650 ts_tablet_manager.cc:1428] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260927 18:39:22.153798 19654 raft_consensus.cc:348] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43837 } } peers { permanent_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 37335 } } peers { permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 } }
I20260927 18:39:22.153916 19654 raft_consensus.cc:374] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:22.153937 19654 raft_consensus.cc:729] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 310dc769e00b4c279134a9d9b0f8bfe9, State: Initialized, Role: FOLLOWER
I20260927 18:39:22.154079 19654 consensus_queue.cc:260] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43837 } } peers { permanent_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 37335 } } peers { permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 } }
I20260927 18:39:22.154366 19654 ts_tablet_manager.cc:1428] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260927 18:39:22.154392 19489 heartbeater.cc:503] Master 127.11.251.254:34227 was elected leader, sending a full tablet report...
W20260927 18:39:22.202980 19491 tablet.cc:2369] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:39:22.231014 19645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.227938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:39555 (local address 127.0.0.1:46198)
0927 18:39:22.228005 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:22.228026 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:22.228076 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:22.228319 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:22.228321 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:22.228323 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:22.228399 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:22.228401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.229146 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.229147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:22.230041 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:22.230044 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.230743 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.230744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:22.230806 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.230810 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:39:22.230854 (+ 44us) client_negotiation.cc:240] Negotiation successful
0927 18:39:22.230915 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 18:39:22.231014 19639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.228075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46198 (local address 127.11.251.252:39555)
0927 18:39:22.228140 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:22.228143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:22.228152 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:39:22.228196 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:22.228197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:22.228221 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:22.228311 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:22.228474 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.229036 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.230127 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.230649 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.230747 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.230857 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:22.230877 (+ 20us) server_negotiation.cc:299] Negotiation successful
0927 18:39:22.230926 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 18:39:22.231635 19182 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260927 18:39:22.232980 19211 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 350e6f81203343e4ae0976f7e03d387e. Latest consensus state: current_term: 1 leader_uuid: "350e6f81203343e4ae0976f7e03d387e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: NON_VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: true } } }
I20260927 18:39:22.233117 19211 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [sys.catalog]: This master's current role is: LEARNER
W20260927 18:39:22.304112 19354 tablet.cc:2369] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:39:22.353206 19627 tablet.cc:2369] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:39:22.457367 19655 raft_consensus.cc:482] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:39:22.457669 19655 raft_consensus.cc:504] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43837 } } peers { permanent_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 37335 } } peers { permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 } }
I20260927 18:39:22.458189 19655 leader_election.cc:288] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 310dc769e00b4c279134a9d9b0f8bfe9 (127.11.251.194:43837), 159be8c77cdc42d1b1a0840730dd6737 (127.11.251.193:36183)
I20260927 18:39:22.461406 19647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.458733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:42601 (local address 127.11.251.193:36183)
0927 18:39:22.458785 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:22.458787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:22.458797 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:39:22.458855 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:22.458856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:22.458870 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:22.458938 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:22.459106 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.459641 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.460648 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.461025 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.461115 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.461126 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:22.461280 (+ 154us) server_negotiation.cc:299] Negotiation successful
0927 18:39:22.461325 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:39:22.461416 19649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.458612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:42975 (local address 127.11.251.194:43837)
0927 18:39:22.458688 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:22.458690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:22.458703 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:39:22.458791 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:22.458792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:22.458806 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:22.458884 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:22.459082 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.459634 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.460648 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.461030 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.461120 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.461130 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:22.461299 (+ 169us) server_negotiation.cc:299] Negotiation successful
0927 18:39:22.461341 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 18:39:22.461423 19629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.458479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:43837 (local address 127.11.251.195:42975)
0927 18:39:22.458604 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:22.458621 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:22.458700 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:22.458898 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:22.458902 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:22.458904 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:22.458998 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:22.459000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.459722 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.459723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:22.460544 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:22.460546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.461120 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.461121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:22.461210 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.461213 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:39:22.461290 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 18:39:22.461333 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":3}
I20260927 18:39:22.461454 19628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.458663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:36183 (local address 127.11.251.195:42601)
0927 18:39:22.458732 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:22.458742 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:22.458786 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:22.458940 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:22.458941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:22.458943 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:22.459034 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:22.459035 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.459739 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.459740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:22.460543 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:22.460545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.461113 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.461114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:22.461195 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.461199 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:39:22.461261 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 18:39:22.461305 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":44}
I20260927 18:39:22.461850 19305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38128a6eeaa34c41a26e77dff92fd714" candidate_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "159be8c77cdc42d1b1a0840730dd6737" is_pre_election: true
I20260927 18:39:22.461855 19442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38128a6eeaa34c41a26e77dff92fd714" candidate_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" is_pre_election: true
I20260927 18:39:22.462044 19305 raft_consensus.cc:2454] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be295e1ae1234a83bb005c34e6a7ad4e in term 0.
I20260927 18:39:22.462044 19442 raft_consensus.cc:2454] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be295e1ae1234a83bb005c34e6a7ad4e in term 0.
I20260927 18:39:22.462281 19511 leader_election.cc:302] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 310dc769e00b4c279134a9d9b0f8bfe9, be295e1ae1234a83bb005c34e6a7ad4e; no voters:
I20260927 18:39:22.462430 19655 raft_consensus.cc:2781] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:39:22.462519 19655 raft_consensus.cc:482] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:39:22.462558 19655 raft_consensus.cc:3037] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:22.463416 19655 raft_consensus.cc:504] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43837 } } peers { permanent_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 37335 } } peers { permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 } }
I20260927 18:39:22.463572 19655 leader_election.cc:288] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e [CANDIDATE]: Term 1 election: Requested vote from peers 310dc769e00b4c279134a9d9b0f8bfe9 (127.11.251.194:43837), 159be8c77cdc42d1b1a0840730dd6737 (127.11.251.193:36183)
I20260927 18:39:22.463742 19442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38128a6eeaa34c41a26e77dff92fd714" candidate_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "310dc769e00b4c279134a9d9b0f8bfe9"
I20260927 18:39:22.463773 19305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38128a6eeaa34c41a26e77dff92fd714" candidate_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "159be8c77cdc42d1b1a0840730dd6737"
I20260927 18:39:22.463832 19442 raft_consensus.cc:3037] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:22.463862 19305 raft_consensus.cc:3037] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:22.464651 19442 raft_consensus.cc:2454] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be295e1ae1234a83bb005c34e6a7ad4e in term 1.
I20260927 18:39:22.464682 19305 raft_consensus.cc:2454] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be295e1ae1234a83bb005c34e6a7ad4e in term 1.
I20260927 18:39:22.464866 19511 leader_election.cc:302] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 159be8c77cdc42d1b1a0840730dd6737, be295e1ae1234a83bb005c34e6a7ad4e; no voters:
I20260927 18:39:22.464973 19655 raft_consensus.cc:2781] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:22.465149 19655 raft_consensus.cc:686] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e [term 1 LEADER]: Becoming Leader. State: Replica: be295e1ae1234a83bb005c34e6a7ad4e, State: Running, Role: LEADER
I20260927 18:39:22.465265 19655 consensus_queue.cc:237] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43837 } } peers { permanent_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 37335 } } peers { permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 } }
I20260927 18:39:22.466648 18388 catalog_manager.cc:5495] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e reported cstate change: term changed from 0 to 1, leader changed from <none> to be295e1ae1234a83bb005c34e6a7ad4e (127.11.251.195). New cstate: current_term: 1 leader_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 37335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 } health_report { overall_health: UNKNOWN } } }
I20260927 18:39:22.467289 18947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "350e6f81203343e4ae0976f7e03d387e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: NON_VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: true } } }
I20260927 18:39:22.467414 18947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:22.468056 18947 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e: attempting to promote NON_VOTER f5cb1a26dafa4b76ab1b5cadcab65903 to VOTER
I20260927 18:39:22.468204 18947 consensus_queue.cc:237] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false } }
I20260927 18:39:22.468729 18997 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 1 FOLLOWER]: Refusing update from remote peer 350e6f81203343e4ae0976f7e03d387e: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260927 18:39:22.468750 19182 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 1 LEARNER]: Refusing update from remote peer 350e6f81203343e4ae0976f7e03d387e: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260927 18:39:22.468922 19666 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260927 18:39:22.469017 18949 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260927 18:39:22.469797 19664 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, f5cb1a26dafa4b76ab1b5cadcab65903 (127.11.251.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false } } }
I20260927 18:39:22.469971 18997 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, f5cb1a26dafa4b76ab1b5cadcab65903 (127.11.251.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false } } }
I20260927 18:39:22.469995 19182 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, f5cb1a26dafa4b76ab1b5cadcab65903 (127.11.251.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false } } }
I20260927 18:39:22.470898 19664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "350e6f81203343e4ae0976f7e03d387e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false } } }
I20260927 18:39:22.470974 19664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:22.470947 19667 sys_catalog.cc:455] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "350e6f81203343e4ae0976f7e03d387e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false } } }
I20260927 18:39:22.471041 19667 sys_catalog.cc:458] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:39:22.471100 18947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "350e6f81203343e4ae0976f7e03d387e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false } } }
I20260927 18:39:22.471124 18948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "350e6f81203343e4ae0976f7e03d387e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false } } }
I20260927 18:39:22.471194 18948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:22.471168 18947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:22.471359 19211 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "350e6f81203343e4ae0976f7e03d387e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false } } }
I20260927 18:39:22.471462 19211 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:39:22.490415 19648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.488061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55906 (local address 127.11.251.195:37335)
0927 18:39:22.488127 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:22.488129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:22.488137 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:39:22.488174 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:22.488175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:22.488192 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:22.488256 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:39:22.488420 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.488916 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.489708 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.489897 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.489952 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.490076 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:39:22.490196 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:39:22.490261 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:22.490333 (+ 72us) server_negotiation.cc:299] Negotiation successful
0927 18:39:22.490351 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:39:22.492537 19305 raft_consensus.cc:1261] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737 [term 1 FOLLOWER]: Refusing update from remote peer be295e1ae1234a83bb005c34e6a7ad4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:22.492600 19442 raft_consensus.cc:1261] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 1 FOLLOWER]: Refusing update from remote peer be295e1ae1234a83bb005c34e6a7ad4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:22.492925 19655 consensus_queue.cc:1035] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:22.493124 19662 consensus_queue.cc:1035] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:22.497134 19671 mvcc.cc:205] Tried to move back new op lower bound from 7334028748766310400 to 7334028748658343936. Current Snapshot: MvccSnapshot[applied={T|T < 7334028748766310400}]
I20260927 18:39:22.498502 18388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:22.498624 18388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:22.500137 18388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 350e6f81203343e4ae0976f7e03d387e: Sending DeleteTablet for 3 replicas of tablet 38128a6eeaa34c41a26e77dff92fd714
I20260927 18:39:22.500480 19422 tablet_service.cc:1514] Processing DeleteTablet for tablet 38128a6eeaa34c41a26e77dff92fd714 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:22 UTC) from {username='oryx', principal='oryx/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:40668
I20260927 18:39:22.500737 19678 tablet_replica.cc:330] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9: stopping tablet replica
I20260927 18:39:22.500927 19558 tablet_service.cc:1514] Processing DeleteTablet for tablet 38128a6eeaa34c41a26e77dff92fd714 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:22 UTC) from {username='oryx', principal='oryx/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:55894
I20260927 18:39:22.500839 19678 raft_consensus.cc:2229] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:22.501206 19678 raft_consensus.cc:2258] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:22.501238 19285 tablet_service.cc:1514] Processing DeleteTablet for tablet 38128a6eeaa34c41a26e77dff92fd714 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:22 UTC) from {username='oryx', principal='oryx/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:34110
I20260927 18:39:22.501410 19681 tablet_replica.cc:330] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737: stopping tablet replica
I20260927 18:39:22.501502 19681 raft_consensus.cc:2229] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:22.501631 19681 raft_consensus.cc:2258] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:22.501682 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 18356
I20260927 18:39:22.502740 19681 ts_tablet_manager.cc:1905] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:22.503364 19680 tablet_replica.cc:330] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e: stopping tablet replica
I20260927 18:39:22.504659 19681 ts_tablet_manager.cc:1918] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:39:22.504738 19681 log.cc:1196] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-0/wal/wals/38128a6eeaa34c41a26e77dff92fd714
I20260927 18:39:22.504870 19680 raft_consensus.cc:2229] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:39:22.505021 19681 ts_tablet_manager.cc:1939] T 38128a6eeaa34c41a26e77dff92fd714 P 159be8c77cdc42d1b1a0840730dd6737: Deleting consensus metadata
I20260927 18:39:22.505034 19680 raft_consensus.cc:2258] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:22.506037 19678 ts_tablet_manager.cc:1905] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:22.507974 19678 ts_tablet_manager.cc:1918] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:39:22.508054 19678 log.cc:1196] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/38128a6eeaa34c41a26e77dff92fd714
I20260927 18:39:22.508355 19678 ts_tablet_manager.cc:1939] T 38128a6eeaa34c41a26e77dff92fd714 P 310dc769e00b4c279134a9d9b0f8bfe9: Deleting consensus metadata
I20260927 18:39:22.509264 19680 ts_tablet_manager.cc:1905] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 18:39:22.510934 19240 connection.cc:535] server connection from 127.0.0.1:34110 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 18:39:22.511310 19680 ts_tablet_manager.cc:1918] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:39:22.511375 19680 log.cc:1196] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/38128a6eeaa34c41a26e77dff92fd714
W20260927 18:39:22.511360 19377 connection.cc:535] server connection from 127.0.0.1:40668 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 18:39:22.511523 19639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.501032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46202 (local address 127.11.251.252:39555)
0927 18:39:22.501076 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:22.501078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:22.502193 (+ 1115us) server_negotiation.cc:407] Connection header received
0927 18:39:22.502288 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:22.502290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:22.502302 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:22.502333 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:22.503524 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.504290 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.505938 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.506226 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.506274 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.508203 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:22.508205 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:22.508205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:22.508213 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:22.510105 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:22.510534 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:22.510536 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:22.510537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:22.510612 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:22.510824 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:22.510825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:22.510826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:22.510954 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:22.511032 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:22.511312 (+ 280us) server_negotiation.cc:299] Negotiation successful
0927 18:39:22.511400 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:39:22.511641 19680 ts_tablet_manager.cc:1939] T 38128a6eeaa34c41a26e77dff92fd714 P be295e1ae1234a83bb005c34e6a7ad4e: Deleting consensus metadata
W20260927 18:39:22.511994 19511 connection.cc:441] server connection from 127.0.0.1:55894 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55894 (request call id 1) could send its response
I20260927 18:39:22.521333 19639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.513745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46210 (local address 127.11.251.252:39555)
0927 18:39:22.513788 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:22.513790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:22.513918 (+ 128us) server_negotiation.cc:407] Connection header received
0927 18:39:22.514007 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:22.514008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:22.514017 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:22.514047 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:22.514933 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.515640 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.516401 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.516619 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.516705 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.518095 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:22.518096 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:22.518097 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:22.518104 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:22.519882 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:22.520281 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:22.520282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:22.520283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:22.520341 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:22.520560 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:22.520561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:22.520561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:22.520710 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:22.520821 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:22.521053 (+ 232us) server_negotiation.cc:299] Negotiation successful
0927 18:39:22.521229 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 18:39:22.522054 19638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:22.513850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50384 (local address 127.11.251.253:41041)
0927 18:39:22.513906 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:22.513908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:22.513917 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:39:22.513944 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:22.513945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:22.513959 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:22.514018 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:22.514736 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.515574 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.516729 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:22.517067 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:22.517106 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:22.518601 (+ 1495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:22.518602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:22.518602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:22.518610 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:22.520517 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:22.520896 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:22.520897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:22.520897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:22.520961 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:22.521198 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:22.521199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:22.521200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:22.521325 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:22.521439 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:22.521841 (+ 402us) server_negotiation.cc:299] Negotiation successful
0927 18:39:22.521947 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:39:23.497835 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:23.497427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.195:38071)
0927 18:39:23.497644 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:23.497755 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":98,"threads_started":1}
I20260927 18:39:23.498062 19690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:23.497657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.193:45309)
0927 18:39:23.497888 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:23.497996 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":71,"threads_started":1}
W20260927 18:39:23.498106 19625 heartbeater.cc:644] Failed to heartbeat to 127.11.251.254:34227 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
W20260927 18:39:23.498277 19352 heartbeater.cc:644] Failed to heartbeat to 127.11.251.254:34227 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
I20260927 18:39:23.498901 19691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:23.498623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.194:36537)
0927 18:39:23.498760 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:23.498840 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
W20260927 18:39:23.499154 19489 heartbeater.cc:644] Failed to heartbeat to 127.11.251.254:34227 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
I20260927 18:39:24.000712 19692 raft_consensus.cc:482] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 350e6f81203343e4ae0976f7e03d387e)
I20260927 18:39:24.000826 19693 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 350e6f81203343e4ae0976f7e03d387e)
I20260927 18:39:24.000919 19692 raft_consensus.cc:504] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false } }
I20260927 18:39:24.001044 19693 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false } }
I20260927 18:39:24.001322 19692 leader_election.cc:288] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 350e6f81203343e4ae0976f7e03d387e (127.11.251.254:34227), f5cb1a26dafa4b76ab1b5cadcab65903 (127.11.251.252:39555)
I20260927 18:39:24.001363 19693 leader_election.cc:288] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 350e6f81203343e4ae0976f7e03d387e (127.11.251.254:34227), fda725da13614957a67aa50a78c8770e (127.11.251.253:41041)
I20260927 18:39:24.001765 19694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:24.001445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.0.0.1:54514)
0927 18:39:24.001648 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:24.001718 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":119,"threads_started":1}
I20260927 18:39:24.001837 19696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:24.001549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.0.0.1:54518)
0927 18:39:24.001727 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:24.001786 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":103,"threads_started":1}
W20260927 18:39:24.001986 19159 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
W20260927 18:39:24.001992 18974 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
I20260927 18:39:24.002731 19696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:24.002577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.0.0.1:54534)
0927 18:39:24.002642 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:24.002682 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260927 18:39:24.002820 19159 leader_election.cc:334] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 350e6f81203343e4ae0976f7e03d387e (127.11.251.254:34227): Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
I20260927 18:39:24.002601 19695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:24.002488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.0.0.1:54526)
0927 18:39:24.002540 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:24.002570 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260927 18:39:24.003548 18974 leader_election.cc:334] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 350e6f81203343e4ae0976f7e03d387e (127.11.251.254:34227): Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
I20260927 18:39:24.005952 19694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:24.001453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:39555 (local address 127.0.0.1:46222)
0927 18:39:24.001944 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:24.001957 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:24.002014 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:24.002307 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:24.002309 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:24.002313 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:24.002390 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:24.002393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:24.003455 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:24.003456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:24.004802 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:24.004805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:24.005659 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:24.005660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:24.005734 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:24.005740 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:39:24.005808 (+ 68us) client_negotiation.cc:240] Negotiation successful
0927 18:39:24.005857 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":66,"threads_started":1}
I20260927 18:39:24.005982 19697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:24.001573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:41041 (local address 127.0.0.1:50394)
0927 18:39:24.001995 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:24.002008 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:24.002111 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:24.002349 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:24.002350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:24.002353 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:24.002427 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:24.002429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:24.003452 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:24.003453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:24.004802 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:24.004805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:24.005659 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:24.005660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:24.005734 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:24.005739 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:39:24.005808 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 18:39:24.005857 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":67,"threads_started":1}
I20260927 18:39:24.005986 19698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:24.001683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50394 (local address 127.11.251.253:41041)
0927 18:39:24.001913 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:24.001916 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:24.002081 (+ 165us) server_negotiation.cc:407] Connection header received
0927 18:39:24.002200 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:24.002201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:24.002214 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:24.002265 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:24.002521 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:24.003362 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:24.004924 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:24.005534 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:24.005652 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:24.005663 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:24.005834 (+ 171us) server_negotiation.cc:299] Negotiation successful
0927 18:39:24.005889 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":58,"threads_started":1}
I20260927 18:39:24.006359 18997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fda725da13614957a67aa50a78c8770e" is_pre_election: true
I20260927 18:39:24.006480 18997 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5cb1a26dafa4b76ab1b5cadcab65903 in term 1.
I20260927 18:39:24.005992 19699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:24.001798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46222 (local address 127.11.251.252:39555)
0927 18:39:24.002151 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:24.002154 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:24.002162 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:39:24.002190 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:24.002191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:24.002202 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:24.002298 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:24.002475 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:24.003303 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:24.004923 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:24.005557 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:24.005659 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:24.005668 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:24.005834 (+ 166us) server_negotiation.cc:299] Negotiation successful
0927 18:39:24.005887 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":46,"threads_started":1}
I20260927 18:39:24.006680 19158 leader_election.cc:302] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f5cb1a26dafa4b76ab1b5cadcab65903, fda725da13614957a67aa50a78c8770e; no voters: 350e6f81203343e4ae0976f7e03d387e
I20260927 18:39:24.006767 19182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fda725da13614957a67aa50a78c8770e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" is_pre_election: true
I20260927 18:39:24.006827 19693 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260927 18:39:24.006888 19182 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fda725da13614957a67aa50a78c8770e in term 1.
I20260927 18:39:24.006925 19693 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 350e6f81203343e4ae0976f7e03d387e)
I20260927 18:39:24.006954 19693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 1 FOLLOWER]: Advancing to term 2
I20260927 18:39:24.007066 18972 leader_election.cc:302] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f5cb1a26dafa4b76ab1b5cadcab65903, fda725da13614957a67aa50a78c8770e; no voters: 350e6f81203343e4ae0976f7e03d387e
I20260927 18:39:24.007195 19692 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260927 18:39:24.007242 19692 raft_consensus.cc:482] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 350e6f81203343e4ae0976f7e03d387e)
I20260927 18:39:24.007262 19692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 1 FOLLOWER]: Advancing to term 2
I20260927 18:39:24.007990 19693 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false } }
I20260927 18:39:24.008153 19693 leader_election.cc:288] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [CANDIDATE]: Term 2 election: Requested vote from peers 350e6f81203343e4ae0976f7e03d387e (127.11.251.254:34227), fda725da13614957a67aa50a78c8770e (127.11.251.253:41041)
I20260927 18:39:24.008132 19692 raft_consensus.cc:504] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false } }
I20260927 18:39:24.008266 19692 leader_election.cc:288] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [CANDIDATE]: Term 2 election: Requested vote from peers 350e6f81203343e4ae0976f7e03d387e (127.11.251.254:34227), f5cb1a26dafa4b76ab1b5cadcab65903 (127.11.251.252:39555)
I20260927 18:39:24.008464 19694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:24.008347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.0.0.1:54548)
0927 18:39:24.008391 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:24.008420 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260927 18:39:24.008407 18997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fda725da13614957a67aa50a78c8770e"
I20260927 18:39:24.008484 19182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fda725da13614957a67aa50a78c8770e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903"
I20260927 18:39:24.008555 18997 raft_consensus.cc:2379] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f5cb1a26dafa4b76ab1b5cadcab65903 in current term 2: Already voted for candidate fda725da13614957a67aa50a78c8770e in this term.
I20260927 18:39:24.008589 19182 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fda725da13614957a67aa50a78c8770e in current term 2: Already voted for candidate f5cb1a26dafa4b76ab1b5cadcab65903 in this term.
I20260927 18:39:24.008747 19694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:24.008673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.0.0.1:54574)
0927 18:39:24.008707 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:24.008728 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260927 18:39:24.008767 19697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:24.008403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.0.0.1:54564)
0927 18:39:24.008443 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:24.008474 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260927 18:39:24.008822 18974 leader_election.cc:334] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 350e6f81203343e4ae0976f7e03d387e (127.11.251.254:34227): Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
I20260927 18:39:24.008854 18974 leader_election.cc:302] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fda725da13614957a67aa50a78c8770e; no voters: 350e6f81203343e4ae0976f7e03d387e, f5cb1a26dafa4b76ab1b5cadcab65903
I20260927 18:39:24.008993 19692 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260927 18:39:24.009157 19697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:24.009042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.0.0.1:54588)
0927 18:39:24.009113 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:24.009137 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260927 18:39:24.009246 19159 leader_election.cc:334] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 350e6f81203343e4ae0976f7e03d387e (127.11.251.254:34227): Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
I20260927 18:39:24.009289 19159 leader_election.cc:302] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5cb1a26dafa4b76ab1b5cadcab65903; no voters: 350e6f81203343e4ae0976f7e03d387e, fda725da13614957a67aa50a78c8770e
I20260927 18:39:24.009369 19693 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260927 18:39:24.498813 19702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:24.498492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.195:41579)
0927 18:39:24.498676 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:24.498723 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":97,"threads_started":1}
I20260927 18:39:24.499163 19703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:24.498771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.193:54549)
0927 18:39:24.499045 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:24.499094 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":118,"threads_started":1}
I20260927 18:39:24.499764 19704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:24.499502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.194:51281)
0927 18:39:24.499664 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:24.499713 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":70,"threads_started":1}
I20260927 18:39:25.500065 19706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.499747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.195:47339)
0927 18:39:25.499954 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.500007 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":115,"threads_started":1}
I20260927 18:39:25.500134 19705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.499732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.193:55079)
0927 18:39:25.500011 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.500063 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":77,"threads_started":1}
I20260927 18:39:25.500501 19707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.500170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.194:55371)
0927 18:39:25.500408 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.500448 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":59,"threads_started":1}
I20260927 18:39:25.629494 19708 raft_consensus.cc:482] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:39:25.629652 19708 raft_consensus.cc:504] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false } }
I20260927 18:39:25.629887 19708 leader_election.cc:288] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 350e6f81203343e4ae0976f7e03d387e (127.11.251.254:34227), f5cb1a26dafa4b76ab1b5cadcab65903 (127.11.251.252:39555)
I20260927 18:39:25.630164 19182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fda725da13614957a67aa50a78c8770e" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" is_pre_election: true
I20260927 18:39:25.630288 19182 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fda725da13614957a67aa50a78c8770e in term 2.
I20260927 18:39:25.630368 19709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.630086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.0.0.1:54822)
0927 18:39:25.630225 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.630285 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":74,"threads_started":1}
I20260927 18:39:25.630474 18972 leader_election.cc:302] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f5cb1a26dafa4b76ab1b5cadcab65903, fda725da13614957a67aa50a78c8770e; no voters:
I20260927 18:39:25.630574 19708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260927 18:39:25.630625 19708 raft_consensus.cc:482] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:39:25.630646 19708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 2 FOLLOWER]: Advancing to term 3
I20260927 18:39:25.630923 19709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.630771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.0.0.1:54836)
0927 18:39:25.630847 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.630883 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260927 18:39:25.631018 18974 leader_election.cc:334] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 350e6f81203343e4ae0976f7e03d387e (127.11.251.254:34227): Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
I20260927 18:39:25.631664 19708 raft_consensus.cc:504] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false } }
I20260927 18:39:25.631843 19708 leader_election.cc:288] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [CANDIDATE]: Term 3 election: Requested vote from peers 350e6f81203343e4ae0976f7e03d387e (127.11.251.254:34227), f5cb1a26dafa4b76ab1b5cadcab65903 (127.11.251.252:39555)
I20260927 18:39:25.632023 19709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.631924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.0.0.1:54838)
0927 18:39:25.631966 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.631998 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260927 18:39:25.632030 19182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fda725da13614957a67aa50a78c8770e" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903"
I20260927 18:39:25.632105 19182 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 2 FOLLOWER]: Advancing to term 3
I20260927 18:39:25.632370 19709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.632276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.0.0.1:54846)
0927 18:39:25.632326 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.632350 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260927 18:39:25.632463 18974 leader_election.cc:334] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 350e6f81203343e4ae0976f7e03d387e (127.11.251.254:34227): Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
I20260927 18:39:25.633019 19182 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fda725da13614957a67aa50a78c8770e in term 3.
I20260927 18:39:25.633222 18972 leader_election.cc:302] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f5cb1a26dafa4b76ab1b5cadcab65903, fda725da13614957a67aa50a78c8770e; no voters: 350e6f81203343e4ae0976f7e03d387e
I20260927 18:39:25.633332 19708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 3 FOLLOWER]: Leader election won for term 3
I20260927 18:39:25.633446 19708 raft_consensus.cc:686] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [term 3 LEADER]: Becoming Leader. State: Replica: fda725da13614957a67aa50a78c8770e, State: Running, Role: LEADER
I20260927 18:39:25.633517 19708 consensus_queue.cc:237] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false } }
I20260927 18:39:25.633920 19711 sys_catalog.cc:455] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [sys.catalog]: SysCatalogTable state changed. Reason: New leader fda725da13614957a67aa50a78c8770e. Latest consensus state: current_term: 3 leader_uuid: "fda725da13614957a67aa50a78c8770e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false } } }
I20260927 18:39:25.634009 19711 sys_catalog.cc:458] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:25.634207 19713 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:39:25.634740 19713 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:39:25.634842 19713 catalog_manager.cc:1259] Loaded cluster ID: 08175ddff41c4f508594ea58d66b74dc
I20260927 18:39:25.634897 19713 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:39:25.635097 19713 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:39:25.635198 19713 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: Loaded TSK: 0
I20260927 18:39:25.635371 19713 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:39:25.685771 18987 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:39:25.687522 18987 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:39:25.688699 19709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.688537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.0.0.1:54856)
0927 18:39:25.688624 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.688668 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260927 18:39:25.688700 19182 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [term 3 FOLLOWER]: Refusing update from remote peer fda725da13614957a67aa50a78c8770e: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260927 18:39:25.688997 19711 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260927 18:39:25.689098 19709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.688970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.0.0.1:54866)
0927 18:39:25.689006 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.689036 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260927 18:39:25.689224 18974 consensus_peers.cc:487] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e -> Peer 350e6f81203343e4ae0976f7e03d387e (127.11.251.254:34227): Couldn't send request to peer 350e6f81203343e4ae0976f7e03d387e. Status: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260927 18:39:25.689765 19715 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fda725da13614957a67aa50a78c8770e. Latest consensus state: current_term: 3 leader_uuid: "fda725da13614957a67aa50a78c8770e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false } } }
I20260927 18:39:25.689898 19715 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:39:25.690423 19716 mvcc.cc:205] Tried to move back new op lower bound from 7334028761858338816 to 7334028761635655680. Current Snapshot: MvccSnapshot[applied={T|T < 7334028761858338816}]
I20260927 18:39:25.690389 19708 sys_catalog.cc:455] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "fda725da13614957a67aa50a78c8770e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false } } }
I20260927 18:39:25.690495 19708 sys_catalog.cc:458] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:25.690546 19715 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "fda725da13614957a67aa50a78c8770e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "350e6f81203343e4ae0976f7e03d387e" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34227 } } peers { permanent_uuid: "fda725da13614957a67aa50a78c8770e" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 41041 } attrs { promote: false } } peers { permanent_uuid: "f5cb1a26dafa4b76ab1b5cadcab65903" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 39555 } attrs { promote: false } } }
I20260927 18:39:25.690627 19715 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5cb1a26dafa4b76ab1b5cadcab65903 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:39:25.696519 19709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.692905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:37335 (local address 127.0.0.1:55914)
0927 18:39:25.692955 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.692969 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:25.693152 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:25.693297 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:25.693297 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:25.693299 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:25.693357 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:25.693358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.694353 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.694354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:25.695532 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:25.695534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.696277 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.696277 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:25.696337 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:25.696340 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:39:25.696402 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 18:39:25.696451 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260927 18:39:25.696521 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.693056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55914 (local address 127.11.251.195:37335)
0927 18:39:25.693173 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:25.693175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:25.693184 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:39:25.693215 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:25.693216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:25.693229 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:25.693296 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:25.693436 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.694266 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.695619 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.696212 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.696281 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:25.696291 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:25.696407 (+ 116us) server_negotiation.cc:299] Negotiation successful
0927 18:39:25.696451 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":61,"threads_started":1}
I20260927 18:39:25.696624 19720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.692944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:36183 (local address 127.0.0.1:34120)
0927 18:39:25.693089 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.693101 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:25.693157 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:25.693617 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:25.693618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:25.693621 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:25.693658 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:25.693660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.694641 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.694642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:25.695706 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:25.695708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.696389 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.696390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:25.696458 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:25.696464 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:39:25.696494 (+ 30us) client_negotiation.cc:240] Negotiation successful
0927 18:39:25.696545 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":43,"thread_start_us":119,"threads_started":1}
I20260927 18:39:25.696976 19721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.693029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34120 (local address 127.11.251.193:36183)
0927 18:39:25.693374 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:25.693385 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:25.693394 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:39:25.693423 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:25.693429 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:25.693440 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:25.693770 (+ 330us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:25.693812 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.694525 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.695798 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.696290 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.696798 (+ 508us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:25.696813 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:25.696834 (+ 21us) server_negotiation.cc:299] Negotiation successful
0927 18:39:25.696896 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":85,"threads_started":1}
I20260927 18:39:25.697224 19558 tablet_service.cc:1467] Processing CreateTablet for tablet b58e3e23b9e54b21a53a7769018f2530 (DEFAULT_TABLE table=test-table [id=3dd4dd2575784be0b987e3b0d15e8b75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:25.697256 19285 tablet_service.cc:1467] Processing CreateTablet for tablet b58e3e23b9e54b21a53a7769018f2530 (DEFAULT_TABLE table=test-table [id=3dd4dd2575784be0b987e3b0d15e8b75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:25.697315 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.693187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:43837 (local address 127.0.0.1:40676)
0927 18:39:25.693289 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.693298 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:25.693345 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:25.693836 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:25.693837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:25.693840 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:25.693904 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:25.693906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.694726 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.694726 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:25.696112 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:25.696113 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.696911 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.696912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:25.696978 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:25.696983 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:39:25.697039 (+ 56us) client_negotiation.cc:240] Negotiation successful
0927 18:39:25.697223 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":23,"thread_start_us":69,"threads_started":1}
I20260927 18:39:25.697397 19558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b58e3e23b9e54b21a53a7769018f2530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:25.697397 19285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b58e3e23b9e54b21a53a7769018f2530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:25.697701 19724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.693482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40676 (local address 127.11.251.194:43837)
0927 18:39:25.693635 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:25.693637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:25.693645 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:39:25.693676 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:25.693677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:25.693713 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:25.693821 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:25.693986 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.694646 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.696206 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.696819 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.696893 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:25.696901 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:25.697513 (+ 612us) server_negotiation.cc:299] Negotiation successful
0927 18:39:25.697626 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":71,"threads_started":1}
I20260927 18:39:25.698040 19422 tablet_service.cc:1467] Processing CreateTablet for tablet b58e3e23b9e54b21a53a7769018f2530 (DEFAULT_TABLE table=test-table [id=3dd4dd2575784be0b987e3b0d15e8b75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:25.698158 19422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b58e3e23b9e54b21a53a7769018f2530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:25.699275 19726 tablet_bootstrap.cc:492] T b58e3e23b9e54b21a53a7769018f2530 P be295e1ae1234a83bb005c34e6a7ad4e: Bootstrap starting.
I20260927 18:39:25.699541 19727 tablet_bootstrap.cc:492] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9: Bootstrap starting.
I20260927 18:39:25.699715 19726 tablet_bootstrap.cc:654] T b58e3e23b9e54b21a53a7769018f2530 P be295e1ae1234a83bb005c34e6a7ad4e: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:25.699951 19727 tablet_bootstrap.cc:654] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:25.700043 19725 tablet_bootstrap.cc:492] T b58e3e23b9e54b21a53a7769018f2530 P 159be8c77cdc42d1b1a0840730dd6737: Bootstrap starting.
I20260927 18:39:25.700368 19726 tablet_bootstrap.cc:492] T b58e3e23b9e54b21a53a7769018f2530 P be295e1ae1234a83bb005c34e6a7ad4e: No bootstrap required, opened a new log
I20260927 18:39:25.700461 19726 ts_tablet_manager.cc:1397] T b58e3e23b9e54b21a53a7769018f2530 P be295e1ae1234a83bb005c34e6a7ad4e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:25.700460 19725 tablet_bootstrap.cc:654] T b58e3e23b9e54b21a53a7769018f2530 P 159be8c77cdc42d1b1a0840730dd6737: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:25.700625 19727 tablet_bootstrap.cc:492] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9: No bootstrap required, opened a new log
I20260927 18:39:25.700677 19727 ts_tablet_manager.cc:1397] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:25.700618 19726 raft_consensus.cc:348] T b58e3e23b9e54b21a53a7769018f2530 P be295e1ae1234a83bb005c34e6a7ad4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 37335 } } peers { permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 } } peers { permanent_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43837 } }
I20260927 18:39:25.700711 19726 raft_consensus.cc:374] T b58e3e23b9e54b21a53a7769018f2530 P be295e1ae1234a83bb005c34e6a7ad4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:25.700737 19726 raft_consensus.cc:729] T b58e3e23b9e54b21a53a7769018f2530 P be295e1ae1234a83bb005c34e6a7ad4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be295e1ae1234a83bb005c34e6a7ad4e, State: Initialized, Role: FOLLOWER
I20260927 18:39:25.700799 19726 consensus_queue.cc:260] T b58e3e23b9e54b21a53a7769018f2530 P be295e1ae1234a83bb005c34e6a7ad4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 37335 } } peers { permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 } } peers { permanent_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43837 } }
I20260927 18:39:25.700850 19727 raft_consensus.cc:348] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 37335 } } peers { permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 } } peers { permanent_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43837 } }
I20260927 18:39:25.700940 19727 raft_consensus.cc:374] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:25.700960 19727 raft_consensus.cc:729] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 310dc769e00b4c279134a9d9b0f8bfe9, State: Initialized, Role: FOLLOWER
I20260927 18:39:25.700987 19726 ts_tablet_manager.cc:1428] T b58e3e23b9e54b21a53a7769018f2530 P be295e1ae1234a83bb005c34e6a7ad4e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 18:39:25.701005 19727 consensus_queue.cc:260] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 37335 } } peers { permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 } } peers { permanent_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43837 } }
I20260927 18:39:25.701192 19727 ts_tablet_manager.cc:1428] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 18:39:25.701743 19707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.701646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.194:44195)
0927 18:39:25.701682 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.701712 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260927 18:39:25.701748 19706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.701336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.195:38013)
0927 18:39:25.701699 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.701724 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343}
I20260927 18:39:25.702049 19725 tablet_bootstrap.cc:492] T b58e3e23b9e54b21a53a7769018f2530 P 159be8c77cdc42d1b1a0840730dd6737: No bootstrap required, opened a new log
I20260927 18:39:25.702174 19707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.702073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.194:38811)
0927 18:39:25.702118 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.702153 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260927 18:39:25.702337 19725 ts_tablet_manager.cc:1397] T b58e3e23b9e54b21a53a7769018f2530 P 159be8c77cdc42d1b1a0840730dd6737: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:39:25.702543 19725 raft_consensus.cc:348] T b58e3e23b9e54b21a53a7769018f2530 P 159be8c77cdc42d1b1a0840730dd6737 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 37335 } } peers { permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 } } peers { permanent_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43837 } }
I20260927 18:39:25.702638 19725 raft_consensus.cc:374] T b58e3e23b9e54b21a53a7769018f2530 P 159be8c77cdc42d1b1a0840730dd6737 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:25.702661 19725 raft_consensus.cc:729] T b58e3e23b9e54b21a53a7769018f2530 P 159be8c77cdc42d1b1a0840730dd6737 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 159be8c77cdc42d1b1a0840730dd6737, State: Initialized, Role: FOLLOWER
I20260927 18:39:25.702726 19725 consensus_queue.cc:260] T b58e3e23b9e54b21a53a7769018f2530 P 159be8c77cdc42d1b1a0840730dd6737 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 37335 } } peers { permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 } } peers { permanent_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43837 } }
I20260927 18:39:25.702965 19725 ts_tablet_manager.cc:1428] T b58e3e23b9e54b21a53a7769018f2530 P 159be8c77cdc42d1b1a0840730dd6737: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:25.703109 19626 heartbeater.cc:503] Master 127.11.251.253:41041 was elected leader, sending a full tablet report...
I20260927 18:39:25.703323 19705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.703214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.193:33399)
0927 18:39:25.703262 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.703291 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260927 18:39:25.703389 19706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.703302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.195:39183)
0927 18:39:25.703341 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.703365 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260927 18:39:25.703747 19353 heartbeater.cc:503] Master 127.11.251.253:41041 was elected leader, sending a full tablet report...
I20260927 18:39:25.915788 19732 raft_consensus.cc:482] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:39:25.915936 19732 raft_consensus.cc:504] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 37335 } } peers { permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 } } peers { permanent_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43837 } }
I20260927 18:39:25.916445 19732 leader_election.cc:288] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be295e1ae1234a83bb005c34e6a7ad4e (127.11.251.195:37335), 159be8c77cdc42d1b1a0840730dd6737 (127.11.251.193:36183)
I20260927 18:39:25.919852 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.916720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:36151 (local address 127.11.251.195:37335)
0927 18:39:25.916777 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:25.916780 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:25.916954 (+ 174us) server_negotiation.cc:407] Connection header received
0927 18:39:25.917039 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:25.917040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:25.917050 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:25.917152 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:25.917362 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.917893 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.919103 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.919472 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.919557 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:25.919568 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:25.919722 (+ 154us) server_negotiation.cc:299] Negotiation successful
0927 18:39:25.919763 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:39:25.919852 19707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.916678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:37335 (local address 127.11.251.194:36151)
0927 18:39:25.916852 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.916866 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:25.916964 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:25.917166 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:25.917167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:25.917171 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:25.917283 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:25.917285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.917986 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.917987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:25.919004 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:25.919006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.919556 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.919557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:25.919643 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:25.919648 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:39:25.919716 (+ 68us) client_negotiation.cc:240] Negotiation successful
0927 18:39:25.919763 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134}
I20260927 18:39:25.919914 19721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.916799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:48455 (local address 127.11.251.193:36183)
0927 18:39:25.916891 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:25.916893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:25.917083 (+ 190us) server_negotiation.cc:407] Connection header received
0927 18:39:25.917176 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:25.917177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:25.917189 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:25.917247 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:25.917421 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.918025 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.919119 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.919499 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.919580 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:25.919594 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:25.919783 (+ 189us) server_negotiation.cc:299] Negotiation successful
0927 18:39:25.919825 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 18:39:25.920003 19734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.916728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:36183 (local address 127.11.251.194:48455)
0927 18:39:25.916968 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.916981 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:25.917104 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:25.917252 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:25.917253 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:25.917255 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:25.917351 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:25.917353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.918111 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.918112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:25.919013 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:25.919015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.919597 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.919598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:25.919701 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:25.919705 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:39:25.919776 (+ 71us) client_negotiation.cc:240] Negotiation successful
0927 18:39:25.919825 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":173,"threads_started":1}
I20260927 18:39:25.920177 19578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b58e3e23b9e54b21a53a7769018f2530" candidate_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" is_pre_election: true
I20260927 18:39:25.920270 19305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b58e3e23b9e54b21a53a7769018f2530" candidate_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "159be8c77cdc42d1b1a0840730dd6737" is_pre_election: true
I20260927 18:39:25.920362 19305 raft_consensus.cc:2454] T b58e3e23b9e54b21a53a7769018f2530 P 159be8c77cdc42d1b1a0840730dd6737 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 310dc769e00b4c279134a9d9b0f8bfe9 in term 0.
I20260927 18:39:25.920382 19578 raft_consensus.cc:2454] T b58e3e23b9e54b21a53a7769018f2530 P be295e1ae1234a83bb005c34e6a7ad4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 310dc769e00b4c279134a9d9b0f8bfe9 in term 0.
I20260927 18:39:25.920576 19375 leader_election.cc:302] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 159be8c77cdc42d1b1a0840730dd6737, 310dc769e00b4c279134a9d9b0f8bfe9; no voters:
I20260927 18:39:25.920791 19732 raft_consensus.cc:2781] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:39:25.920851 19732 raft_consensus.cc:482] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:39:25.920879 19732 raft_consensus.cc:3037] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:25.921873 19732 raft_consensus.cc:504] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 37335 } } peers { permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 } } peers { permanent_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43837 } }
I20260927 18:39:25.922034 19732 leader_election.cc:288] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 [CANDIDATE]: Term 1 election: Requested vote from peers be295e1ae1234a83bb005c34e6a7ad4e (127.11.251.195:37335), 159be8c77cdc42d1b1a0840730dd6737 (127.11.251.193:36183)
I20260927 18:39:25.922210 19305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b58e3e23b9e54b21a53a7769018f2530" candidate_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "159be8c77cdc42d1b1a0840730dd6737"
I20260927 18:39:25.922293 19305 raft_consensus.cc:3037] T b58e3e23b9e54b21a53a7769018f2530 P 159be8c77cdc42d1b1a0840730dd6737 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:25.922339 19578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b58e3e23b9e54b21a53a7769018f2530" candidate_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be295e1ae1234a83bb005c34e6a7ad4e"
I20260927 18:39:25.922428 19578 raft_consensus.cc:3037] T b58e3e23b9e54b21a53a7769018f2530 P be295e1ae1234a83bb005c34e6a7ad4e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:25.923010 19305 raft_consensus.cc:2454] T b58e3e23b9e54b21a53a7769018f2530 P 159be8c77cdc42d1b1a0840730dd6737 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 310dc769e00b4c279134a9d9b0f8bfe9 in term 1.
I20260927 18:39:25.923179 19375 leader_election.cc:302] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 159be8c77cdc42d1b1a0840730dd6737, 310dc769e00b4c279134a9d9b0f8bfe9; no voters:
I20260927 18:39:25.923283 19732 raft_consensus.cc:2781] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:25.923321 19578 raft_consensus.cc:2454] T b58e3e23b9e54b21a53a7769018f2530 P be295e1ae1234a83bb005c34e6a7ad4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 310dc769e00b4c279134a9d9b0f8bfe9 in term 1.
I20260927 18:39:25.923473 19732 raft_consensus.cc:686] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 1 LEADER]: Becoming Leader. State: Replica: 310dc769e00b4c279134a9d9b0f8bfe9, State: Running, Role: LEADER
I20260927 18:39:25.923568 19732 consensus_queue.cc:237] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 37335 } } peers { permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 } } peers { permanent_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43837 } }
I20260927 18:39:25.923733 19490 heartbeater.cc:503] Master 127.11.251.253:41041 was elected leader, sending a full tablet report...
I20260927 18:39:25.924021 19734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.923901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.194:45723)
0927 18:39:25.923952 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.923992 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260927 18:39:25.924607 18986 catalog_manager.cc:5495] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 310dc769e00b4c279134a9d9b0f8bfe9 (127.11.251.194). New cstate: current_term: 1 leader_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 37335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "310dc769e00b4c279134a9d9b0f8bfe9" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43837 } health_report { overall_health: HEALTHY } } }
I20260927 18:39:25.982499 19724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.979722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40680 (local address 127.11.251.194:43837)
0927 18:39:25.979793 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:25.979795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:25.979806 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:39:25.979933 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:25.979934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:25.979952 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:25.980023 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:39:25.980189 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.980780 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.981712 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:25.981905 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:25.981983 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:25.982165 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:39:25.982268 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:39:25.982338 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:25.982424 (+ 86us) server_negotiation.cc:299] Negotiation successful
0927 18:39:25.982443 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:39:25.983831 19305 raft_consensus.cc:1261] T b58e3e23b9e54b21a53a7769018f2530 P 159be8c77cdc42d1b1a0840730dd6737 [term 1 FOLLOWER]: Refusing update from remote peer 310dc769e00b4c279134a9d9b0f8bfe9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:25.983886 19578 raft_consensus.cc:1261] T b58e3e23b9e54b21a53a7769018f2530 P be295e1ae1234a83bb005c34e6a7ad4e [term 1 FOLLOWER]: Refusing update from remote peer 310dc769e00b4c279134a9d9b0f8bfe9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:25.984262 19735 consensus_queue.cc:1035] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be295e1ae1234a83bb005c34e6a7ad4e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 37335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:25.984418 19732 consensus_queue.cc:1035] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "159be8c77cdc42d1b1a0840730dd6737" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:25.985728 19705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.985539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.193:45613)
0927 18:39:25.985659 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.985694 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260927 18:39:25.986057 19706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.985760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.195:56423)
0927 18:39:25.985978 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.986017 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194}
I20260927 18:39:25.986677 19738 mvcc.cc:205] Tried to move back new op lower bound from 7334028763066982400 to 7334028762823356416. Current Snapshot: MvccSnapshot[applied={T|T < 7334028763066982400}]
I20260927 18:39:25.987227 19734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.986983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.194:54151)
0927 18:39:25.987165 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.987196 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260927 18:39:25.989161 18987 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:25.989203 19705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.989081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.193:45023)
0927 18:39:25.989127 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.989164 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260927 18:39:25.989252 18987 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:25.990007 19740 mvcc.cc:205] Tried to move back new op lower bound from 7334028763066982400 to 7334028762823356416. Current Snapshot: MvccSnapshot[applied={T|T < 7334028763066982400}]
I20260927 18:39:25.990700 19706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:25.990537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:34227 (local address 127.11.251.195:47491)
0927 18:39:25.990634 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:25.990664 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260927 18:39:25.990859 18987 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fda725da13614957a67aa50a78c8770e: Sending DeleteTablet for 3 replicas of tablet b58e3e23b9e54b21a53a7769018f2530
I20260927 18:39:25.991168 19558 tablet_service.cc:1514] Processing DeleteTablet for tablet b58e3e23b9e54b21a53a7769018f2530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:25 UTC) from {username='oryx', principal='oryx/127.11.251.253@KRBTEST.COM'} at 127.0.0.1:55914
I20260927 18:39:25.991205 19285 tablet_service.cc:1514] Processing DeleteTablet for tablet b58e3e23b9e54b21a53a7769018f2530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:25 UTC) from {username='oryx', principal='oryx/127.11.251.253@KRBTEST.COM'} at 127.0.0.1:34120
I20260927 18:39:25.991355 19557 tablet_service.cc:1514] Processing DeleteTablet for tablet b58e3e23b9e54b21a53a7769018f2530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:25 UTC) from {username='oryx', principal='oryx/127.11.251.253@KRBTEST.COM'} at 127.0.0.1:55914
I20260927 18:39:25.991482 19422 tablet_service.cc:1514] Processing DeleteTablet for tablet b58e3e23b9e54b21a53a7769018f2530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:25 UTC) from {username='oryx', principal='oryx/127.11.251.253@KRBTEST.COM'} at 127.0.0.1:40676
I20260927 18:39:25.991685 19749 tablet_replica.cc:330] T b58e3e23b9e54b21a53a7769018f2530 P 159be8c77cdc42d1b1a0840730dd6737: stopping tablet replica
I20260927 18:39:25.991726 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 19134
I20260927 18:39:25.997931 19750 tablet_replica.cc:330] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9: stopping tablet replica
I20260927 18:39:25.998025 19750 raft_consensus.cc:2229] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:39:25.998152 19750 raft_consensus.cc:2258] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:25.998291 19748 tablet_replica.cc:330] T b58e3e23b9e54b21a53a7769018f2530 P be295e1ae1234a83bb005c34e6a7ad4e: stopping tablet replica
I20260927 18:39:25.998358 19748 raft_consensus.cc:2229] T b58e3e23b9e54b21a53a7769018f2530 P be295e1ae1234a83bb005c34e6a7ad4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:25.998417 19748 raft_consensus.cc:2258] T b58e3e23b9e54b21a53a7769018f2530 P be295e1ae1234a83bb005c34e6a7ad4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:25.998739 19750 ts_tablet_manager.cc:1905] T b58e3e23b9e54b21a53a7769018f2530 P 310dc769e00b4c279134a9d9b0f8bfe9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:25.998845 19748 ts_tablet_manager.cc:1905] T b58e3e23b9e54b21a53a7769018f2530 P be295e1ae1234a83bb005c34e6a7ad4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:26.000442 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:26.000327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:36183 (local address 127.0.0.1:34126)
0927 18:39:26.000377 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:26.000404 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.193:36183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260927 18:39:26.000633 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 19359
W20260927 18:39:26.000658 18971 catalog_manager.cc:4584] TS 159be8c77cdc42d1b1a0840730dd6737 (127.11.251.193:36183): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b58e3e23b9e54b21a53a7769018f2530: Network error: Client connection negotiation failed: client connection to 127.11.251.193:36183: connect: Connection refused (error 111)
I20260927 18:39:26.001631 19748 ts_tablet_manager.cc:1918] T b58e3e23b9e54b21a53a7769018f2530 P be295e1ae1234a83bb005c34e6a7ad4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:39:26.001701 19748 log.cc:1196] T b58e3e23b9e54b21a53a7769018f2530 P be295e1ae1234a83bb005c34e6a7ad4e: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/b58e3e23b9e54b21a53a7769018f2530
I20260927 18:39:26.001988 19748 ts_tablet_manager.cc:1939] T b58e3e23b9e54b21a53a7769018f2530 P be295e1ae1234a83bb005c34e6a7ad4e: Deleting consensus metadata
I20260927 18:39:26.002472 18973 catalog_manager.cc:4841] TS be295e1ae1234a83bb005c34e6a7ad4e (127.11.251.195:37335): tablet b58e3e23b9e54b21a53a7769018f2530 (table test-table [id=3dd4dd2575784be0b987e3b0d15e8b75]) successfully deleted
W20260927 18:39:26.002796 18973 catalog_manager.cc:4808] TS be295e1ae1234a83bb005c34e6a7ad4e (127.11.251.195:37335): delete failed for tablet b58e3e23b9e54b21a53a7769018f2530 because the tablet was not found. No further retry: Not found: Tablet not found: b58e3e23b9e54b21a53a7769018f2530
I20260927 18:39:26.007488 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:26.007303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:43837 (local address 127.0.0.1:40688)
0927 18:39:26.007412 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:26.007443 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.194:43837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260927 18:39:26.007582 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 19495
I20260927 18:39:26.014564 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 18852
I20260927 18:39:26.021047 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 19041
2026-09-27T18:39:26Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6823 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:39:26 dist-test-slave-z47j krb5kdc[19753](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:39:26 dist-test-slave-z47j krb5kdc[19753](info): set up 2 sockets
Sep 27 18:39:26 dist-test-slave-z47j krb5kdc[19753](info): commencing operation
krb5kdc: starting...
W20260927 18:39:27.094621 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:39:27 dist-test-slave-z47j krb5kdc[19753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:39:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:39:27Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.254.keytab.
Entry for principal oryx/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.254.keytab.
I20260927 18:39:27.220142 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:46095
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.254.keytab
--principal=oryx/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:40597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:46095
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:27.308708 19766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:27.308938 19766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:27.308960 19766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:27.310269 19766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:27.310313 19766 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 18:39:27.310326 19766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:27.310339 19766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:27.310351 19766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:27.310364 19766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:39:27.311933 19766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:40597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:46095
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.254.keytab
--principal=oryx/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:46095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:27.312217 19766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:27.312481 19766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:27.317808 19771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:27.317835 19774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:27.318112 19766 server_base.cc:1034] running on GCE node
W20260927 18:39:27.318164 19772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:27.318430 19766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:27.318787 19766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:27.319963 19766 hybrid_clock.cc:648] HybridClock initialized: now 1790534367319941 us; error 40 us; skew 500 ppm
Sep 27 18:39:27 dist-test-slave-z47j krb5kdc[19753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534367, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:27.322808 19766 init.cc:377] Logged in from keytab as oryx/127.11.251.254@KRBTEST.COM (short username oryx)
I20260927 18:39:27.323235 19766 webserver.cc:460] Webserver started at http://127.11.251.254:42573/ using document root <none> and password file <none>
I20260927 18:39:27.323424 19766 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:27.323463 19766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:27.323565 19766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:27.324708 19766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "464cdece19dc49ca88c32d90fa0a40ad"
format_stamp: "Formatted at 2026-09-27 18:39:27 on dist-test-slave-z47j"
server_key: "b2c50dc8136a17732a03fb64320b0182"
server_key_iv: "ad502919dd7c9658be522f4dd2e96349"
server_key_version: "encryptionkey@0"
I20260927 18:39:27.325111 19766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "464cdece19dc49ca88c32d90fa0a40ad"
format_stamp: "Formatted at 2026-09-27 18:39:27 on dist-test-slave-z47j"
server_key: "b2c50dc8136a17732a03fb64320b0182"
server_key_iv: "ad502919dd7c9658be522f4dd2e96349"
server_key_version: "encryptionkey@0"
I20260927 18:39:27.326648 19766 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:39:27.327478 19781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:27.327698 19766 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:27.327811 19766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "464cdece19dc49ca88c32d90fa0a40ad"
format_stamp: "Formatted at 2026-09-27 18:39:27 on dist-test-slave-z47j"
server_key: "b2c50dc8136a17732a03fb64320b0182"
server_key_iv: "ad502919dd7c9658be522f4dd2e96349"
server_key_version: "encryptionkey@0"
I20260927 18:39:27.327889 19766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:27.342228 19766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:27.344754 19766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:27.344935 19766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:27.350544 19766 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:46095
I20260927 18:39:27.350579 19843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:46095 every 8 connection(s)
I20260927 18:39:27.350966 19766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:39:27.351678 19844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:27.354421 19844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad: Bootstrap starting.
I20260927 18:39:27.355144 19844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:27.355502 19844 log.cc:824] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:27.355507 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 19766
I20260927 18:39:27.355654 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:39:27.355903 12271 external_mini_cluster.cc:1428] Setting key 98ef27e239403d590029d14e18212ba8
I20260927 18:39:27.356257 19844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad: No bootstrap required, opened a new log
I20260927 18:39:27.358734 19844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "464cdece19dc49ca88c32d90fa0a40ad" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46095 } }
I20260927 18:39:27.358919 19844 raft_consensus.cc:374] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:27.358964 19844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 464cdece19dc49ca88c32d90fa0a40ad, State: Initialized, Role: FOLLOWER
I20260927 18:39:27.359122 19844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "464cdece19dc49ca88c32d90fa0a40ad" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46095 } }
I20260927 18:39:27.359217 19844 raft_consensus.cc:388] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:39:27.359263 19844 raft_consensus.cc:482] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:39:27.359335 19844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:27.359948 19844 raft_consensus.cc:504] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "464cdece19dc49ca88c32d90fa0a40ad" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46095 } }
I20260927 18:39:27.360074 19844 leader_election.cc:302] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 464cdece19dc49ca88c32d90fa0a40ad; no voters:
I20260927 18:39:27.360280 19844 leader_election.cc:288] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:39:27.360312 19849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:27.360435 19849 raft_consensus.cc:686] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad [term 1 LEADER]: Becoming Leader. State: Replica: 464cdece19dc49ca88c32d90fa0a40ad, State: Running, Role: LEADER
I20260927 18:39:27.360520 19849 consensus_queue.cc:237] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "464cdece19dc49ca88c32d90fa0a40ad" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46095 } }
I20260927 18:39:27.360572 19844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:27.360888 19849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 464cdece19dc49ca88c32d90fa0a40ad. Latest consensus state: current_term: 1 leader_uuid: "464cdece19dc49ca88c32d90fa0a40ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "464cdece19dc49ca88c32d90fa0a40ad" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46095 } } }
I20260927 18:39:27.360980 19849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:27.360901 19851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "464cdece19dc49ca88c32d90fa0a40ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "464cdece19dc49ca88c32d90fa0a40ad" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46095 } } }
I20260927 18:39:27.361135 19851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:27.361225 19854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:39:27.361908 19854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:39:27.363701 19854 catalog_manager.cc:1347] Generated new cluster ID: 47e5a0f29b4b4fdfa09fcaf381018d2c
I20260927 18:39:27.363775 19854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:39:27 dist-test-slave-z47j krb5kdc[19753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM
I20260927 18:39:27.369457 19854 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:39:27.370157 19854 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:39:27.370790 19848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:27.356519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53666 (local address 127.11.251.254:46095)
0927 18:39:27.356717 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:27.356728 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:27.356766 (+ 38us) server_negotiation.cc:407] Connection header received
0927 18:39:27.358725 (+ 1959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:27.358734 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:27.358792 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:27.358977 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:27.359835 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.361251 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.362391 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.362957 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.363106 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:27.366080 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:27.366085 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:27.366085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:27.366098 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:27.368708 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:27.369104 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:27.369107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:27.369108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:27.369253 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:27.369470 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:27.369471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:27.369471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:27.369771 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:27.369943 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:27.370404 (+ 461us) server_negotiation.cc:299] Negotiation successful
0927 18:39:27.370567 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260927 18:39:27.377816 19854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad: Generated new TSK 0
I20260927 18:39:27.378019 19854 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.251.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.251.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.251.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.193.keytab.
Entry for principal oryx/127.11.251.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.193.keytab.
I20260927 18:39:27.413133 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:46095
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.193.keytab
--principal=oryx/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:40597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:27.507905 19870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:27.508139 19870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:27.508163 19870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:27.509379 19870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:27.509418 19870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:27.509439 19870 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:39:27.509457 19870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:39:27.510982 19870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:40597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.193.keytab
--principal=oryx/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.251.254:46095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:27.511184 19870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:27.511379 19870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:27.516842 19878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:27.516906 19876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:27.516857 19875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:27.517016 19870 server_base.cc:1034] running on GCE node
I20260927 18:39:27.517320 19870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:27.517560 19870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:27.518798 19870 hybrid_clock.cc:648] HybridClock initialized: now 1790534367518786 us; error 51 us; skew 500 ppm
Sep 27 18:39:27 dist-test-slave-z47j krb5kdc[19753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534367, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:27.521700 19870 init.cc:377] Logged in from keytab as oryx/127.11.251.193@KRBTEST.COM (short username oryx)
I20260927 18:39:27.522178 19870 webserver.cc:460] Webserver started at http://127.11.251.193:37295/ using document root <none> and password file <none>
I20260927 18:39:27.522416 19870 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:27.522486 19870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:27.522652 19870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:27.523794 19870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "9eb34beb910b4457a618e130cb7c8b5e"
format_stamp: "Formatted at 2026-09-27 18:39:27 on dist-test-slave-z47j"
server_key: "5027c60a785e19bfcdf246ef5f73b63e"
server_key_iv: "5c5e5610b3f616ed59f4e8006de35fe6"
server_key_version: "encryptionkey@0"
I20260927 18:39:27.524175 19870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "9eb34beb910b4457a618e130cb7c8b5e"
format_stamp: "Formatted at 2026-09-27 18:39:27 on dist-test-slave-z47j"
server_key: "5027c60a785e19bfcdf246ef5f73b63e"
server_key_iv: "5c5e5610b3f616ed59f4e8006de35fe6"
server_key_version: "encryptionkey@0"
I20260927 18:39:27.525645 19870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 18:39:27.526451 19885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:27.526649 19870 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:27.526751 19870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "9eb34beb910b4457a618e130cb7c8b5e"
format_stamp: "Formatted at 2026-09-27 18:39:27 on dist-test-slave-z47j"
server_key: "5027c60a785e19bfcdf246ef5f73b63e"
server_key_iv: "5c5e5610b3f616ed59f4e8006de35fe6"
server_key_version: "encryptionkey@0"
I20260927 18:39:27.526839 19870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:27.536263 19870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:27.538197 19870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:27.538365 19870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:27.539005 19870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:27.539069 19870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:27.539110 19870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:27.539147 19870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:39:27 dist-test-slave-z47j krb5kdc[19753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534367, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM
I20260927 18:39:27.547009 19870 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:42283
I20260927 18:39:27.547065 20000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:42283 every 8 connection(s)
I20260927 18:39:27.547427 19870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:39:27.549093 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 19870
I20260927 18:39:27.549228 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:39:27.549485 12271 external_mini_cluster.cc:1428] Setting key 7a0dec2052743395e7d86cc575599c14
I20260927 18:39:27.550973 19848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:27.539455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:39543 (local address 127.11.251.254:46095)
0927 18:39:27.539528 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:27.539530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:27.539757 (+ 227us) server_negotiation.cc:407] Connection header received
0927 18:39:27.540281 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:27.540282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:27.540299 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:27.540379 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:27.541286 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.542240 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.543337 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.543577 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.543675 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:27.547352 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:27.547355 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:27.547355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:27.547364 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:27.549113 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:27.549703 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:27.549704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:27.549705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:27.549788 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:27.550133 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:27.550134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:27.550135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:27.550322 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:27.550408 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:27.550811 (+ 403us) server_negotiation.cc:299] Negotiation successful
0927 18:39:27.550899 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 18:39:27.551287 19894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:27.539436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:46095 (local address 127.11.251.193:39543)
0927 18:39:27.539585 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:27.539655 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:27.540172 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:27.540454 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:27.540458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:27.540846 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:27.541175 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:27.541182 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.542379 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.542380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:27.543239 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:27.543241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.543684 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.543685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:27.543817 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:27.544347 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:27.544361 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:27.547238 (+ 2877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:27.549239 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:27.549247 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:27.549249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:27.549592 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:27.549892 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:27.549893 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:27.549894 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:27.550050 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:27.550465 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:27.550472 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:27.550645 (+ 173us) client_negotiation.cc:769] Sending connection context
0927 18:39:27.550787 (+ 142us) client_negotiation.cc:240] Negotiation successful
0927 18:39:27.550972 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260927 18:39:27.552747 20001 heartbeater.cc:347] Connected to a master server at 127.11.251.254:46095
I20260927 18:39:27.552855 20001 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:27.553083 20001 heartbeater.cc:511] Master 127.11.251.254:46095 requested a full tablet report, sending...
I20260927 18:39:27.553853 19797 ts_manager.cc:194] Registered new tserver with Master: 9eb34beb910b4457a618e130cb7c8b5e (127.11.251.193:42283)
I20260927 18:39:27.554626 19797 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:39543
WARNING: no policy specified for oryx/127.11.251.194@KRBTEST.COM; defaulting to no policy
I20260927 18:39:27.561295 19848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:27.554027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:38281 (local address 127.11.251.254:46095)
0927 18:39:27.554077 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:27.554080 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:27.554314 (+ 234us) server_negotiation.cc:407] Connection header received
0927 18:39:27.554417 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:27.554418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:27.554432 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:27.554466 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:27.554987 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.555794 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.557006 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.557213 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.557280 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:27.558486 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:27.558488 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:27.558488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:27.558493 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:27.559957 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:27.560380 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:27.560381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:27.560381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:27.560435 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:27.560687 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:27.560688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:27.560688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:27.560838 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:27.560928 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:27.561112 (+ 184us) server_negotiation.cc:299] Negotiation successful
0927 18:39:27.561192 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:39:27.561299 20008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:27.554074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:46095 (local address 127.11.251.193:38281)
0927 18:39:27.554237 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:27.554250 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:27.554341 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:27.554534 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:27.554535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:27.554782 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:27.554875 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:27.554877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.555939 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.555940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:27.556901 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:27.556905 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.557305 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.557307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:27.557392 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:27.557917 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:27.557923 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:27.558395 (+ 472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:27.560044 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:27.560046 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:27.560047 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:27.560300 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:27.560507 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:27.560508 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:27.560509 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:27.560583 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:27.560917 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:27.560918 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:27.560970 (+ 52us) client_negotiation.cc:769] Sending connection context
0927 18:39:27.561059 (+ 89us) client_negotiation.cc:240] Negotiation successful
0927 18:39:27.561183 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.251.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.251.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.194.keytab.
Entry for principal oryx/127.11.251.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.194.keytab.
I20260927 18:39:27.580775 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:46095
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.194.keytab
--principal=oryx/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:40597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:27.672353 20010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:27.672585 20010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:27.672608 20010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:27.673831 20010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:27.673872 20010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:27.673893 20010 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:39:27.673938 20010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:39:27.675493 20010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:40597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.194.keytab
--principal=oryx/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.251.254:46095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:27.675774 20010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:27.676028 20010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:27.681617 20015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:27.681695 20018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:27.681731 20016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:27.681838 20010 server_base.cc:1034] running on GCE node
I20260927 18:39:27.682085 20010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:27.682304 20010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:27.683449 20010 hybrid_clock.cc:648] HybridClock initialized: now 1790534367683422 us; error 52 us; skew 500 ppm
Sep 27 18:39:27 dist-test-slave-z47j krb5kdc[19753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534367, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:27.686318 20010 init.cc:377] Logged in from keytab as oryx/127.11.251.194@KRBTEST.COM (short username oryx)
I20260927 18:39:27.686862 20010 webserver.cc:460] Webserver started at http://127.11.251.194:39673/ using document root <none> and password file <none>
I20260927 18:39:27.687124 20010 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:27.687173 20010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:27.687341 20010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:27.688493 20010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "c73d576dc1304b8181c35475c71a35e4"
format_stamp: "Formatted at 2026-09-27 18:39:27 on dist-test-slave-z47j"
server_key: "cd4188461bf61a8ed0db407b0d95cebd"
server_key_iv: "140868ed347e9b4518cbdbb8e5ca3662"
server_key_version: "encryptionkey@0"
I20260927 18:39:27.688877 20010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "c73d576dc1304b8181c35475c71a35e4"
format_stamp: "Formatted at 2026-09-27 18:39:27 on dist-test-slave-z47j"
server_key: "cd4188461bf61a8ed0db407b0d95cebd"
server_key_iv: "140868ed347e9b4518cbdbb8e5ca3662"
server_key_version: "encryptionkey@0"
I20260927 18:39:27.690410 20010 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:39:27.691218 20025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:27.691395 20010 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:39:27.691495 20010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "c73d576dc1304b8181c35475c71a35e4"
format_stamp: "Formatted at 2026-09-27 18:39:27 on dist-test-slave-z47j"
server_key: "cd4188461bf61a8ed0db407b0d95cebd"
server_key_iv: "140868ed347e9b4518cbdbb8e5ca3662"
server_key_version: "encryptionkey@0"
I20260927 18:39:27.691576 20010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:27.701298 20010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:27.703711 20010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:27.703888 20010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:27.704455 20010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:27.704527 20010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:27.704576 20010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:27.704612 20010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:39:27 dist-test-slave-z47j krb5kdc[19753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534367, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM
I20260927 18:39:27.712785 20010 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:43985
I20260927 18:39:27.712867 20140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:43985 every 8 connection(s)
I20260927 18:39:27.713255 20010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:39:27.715677 19848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:27.705073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:60359 (local address 127.11.251.254:46095)
0927 18:39:27.705162 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:27.705165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:27.705381 (+ 216us) server_negotiation.cc:407] Connection header received
0927 18:39:27.706016 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:27.706018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:27.706036 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:27.706132 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:27.706920 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.707837 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.709479 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.709847 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.709934 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:27.712817 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:27.712818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:27.712819 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:27.712827 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:27.714376 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:27.714793 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:27.714794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:27.714794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:27.714844 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:27.715049 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:27.715049 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:27.715050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:27.715155 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:27.715208 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:27.715510 (+ 302us) server_negotiation.cc:299] Negotiation successful
0927 18:39:27.715575 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 18:39:27.715859 20034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:27.705023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:46095 (local address 127.11.251.194:60359)
0927 18:39:27.705229 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:27.705272 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:27.705903 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:27.706223 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:27.706228 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:27.706610 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:27.706813 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:27.706818 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.707941 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.707942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:27.709376 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:27.709380 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.709947 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.709948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:27.710089 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:27.710605 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:27.710616 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:27.712679 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:27.714468 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:27.714472 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:27.714474 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:27.714726 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:27.714902 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:27.714903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:27.714903 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:27.714996 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:27.715239 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:27.715246 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:27.715379 (+ 133us) client_negotiation.cc:769] Sending connection context
0927 18:39:27.715490 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 18:39:27.715635 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":134,"threads_started":1}
I20260927 18:39:27.716470 20141 heartbeater.cc:347] Connected to a master server at 127.11.251.254:46095
I20260927 18:39:27.716567 20141 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:27.716666 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 20010
I20260927 18:39:27.716781 20141 heartbeater.cc:511] Master 127.11.251.254:46095 requested a full tablet report, sending...
I20260927 18:39:27.716804 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:39:27.717097 12271 external_mini_cluster.cc:1428] Setting key e76ba26c31dc30a4faf16a5127bfe497
I20260927 18:39:27.717212 19798 ts_manager.cc:194] Registered new tserver with Master: c73d576dc1304b8181c35475c71a35e4 (127.11.251.194:43985)
I20260927 18:39:27.717760 19798 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:60359
I20260927 18:39:27.724371 20147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:27.717756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:46095 (local address 127.11.251.194:39583)
0927 18:39:27.718066 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:27.718079 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:27.718148 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:27.718274 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:27.718275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:27.718516 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:27.718602 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:27.718604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.719318 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.719318 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:27.719997 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:27.719999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.720508 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.720509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:27.720555 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:27.721021 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:27.721027 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:27.721332 (+ 305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:27.723283 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:27.723284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:27.723284 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:27.723451 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:27.723687 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:27.723688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:27.723688 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:27.723751 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:27.724142 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:27.724143 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:27.724180 (+ 37us) client_negotiation.cc:769] Sending connection context
0927 18:39:27.724228 (+ 48us) client_negotiation.cc:240] Negotiation successful
0927 18:39:27.724308 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":84,"threads_started":1}
I20260927 18:39:27.724470 19848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:27.717791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:39583 (local address 127.11.251.254:46095)
0927 18:39:27.717830 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:27.717832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:27.718137 (+ 305us) server_negotiation.cc:407] Connection header received
0927 18:39:27.718208 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:27.718209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:27.718220 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:27.718273 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:27.718696 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.719218 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.720081 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.720415 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.720525 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:27.721423 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:27.721424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:27.721425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:27.721432 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:27.723194 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:27.723526 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:27.723528 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:27.723528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:27.723590 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:27.723827 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:27.723828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:27.723828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:27.724057 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:27.724149 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:27.724269 (+ 120us) server_negotiation.cc:299] Negotiation successful
0927 18:39:27.724346 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for oryx/127.11.251.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.251.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.251.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.195.keytab.
Entry for principal oryx/127.11.251.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.195.keytab.
I20260927 18:39:27.745916 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:46095
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.195.keytab
--principal=oryx/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:40597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:27.835369 20150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:27.835615 20150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:27.835649 20150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:27.836858 20150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:27.836899 20150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:27.836920 20150 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:39:27.836937 20150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:39:27.838551 20150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:40597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.195.keytab
--principal=oryx/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.251.254:46095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:27.838819 20150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:27.839066 20150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:27.844388 20155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:27.844393 20158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:27.844486 20150 server_base.cc:1034] running on GCE node
W20260927 18:39:27.844409 20156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:27.844887 20150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:27.845091 20150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:27.846225 20150 hybrid_clock.cc:648] HybridClock initialized: now 1790534367846206 us; error 38 us; skew 500 ppm
Sep 27 18:39:27 dist-test-slave-z47j krb5kdc[19753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534367, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:27.848909 20150 init.cc:377] Logged in from keytab as oryx/127.11.251.195@KRBTEST.COM (short username oryx)
I20260927 18:39:27.849367 20150 webserver.cc:460] Webserver started at http://127.11.251.195:38635/ using document root <none> and password file <none>
I20260927 18:39:27.849663 20150 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:27.849714 20150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:27.849854 20150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:27.851050 20150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "87bc6c3807a64aa2891412273b2eb0fd"
format_stamp: "Formatted at 2026-09-27 18:39:27 on dist-test-slave-z47j"
server_key: "32f8ee6f31019234e67113173677d18f"
server_key_iv: "b913885865c719550732bfbf5148b900"
server_key_version: "encryptionkey@0"
I20260927 18:39:27.851481 20150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "87bc6c3807a64aa2891412273b2eb0fd"
format_stamp: "Formatted at 2026-09-27 18:39:27 on dist-test-slave-z47j"
server_key: "32f8ee6f31019234e67113173677d18f"
server_key_iv: "b913885865c719550732bfbf5148b900"
server_key_version: "encryptionkey@0"
I20260927 18:39:27.853070 20150 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:39:27.853999 20165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:27.854223 20150 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 18:39:27.854326 20150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "87bc6c3807a64aa2891412273b2eb0fd"
format_stamp: "Formatted at 2026-09-27 18:39:27 on dist-test-slave-z47j"
server_key: "32f8ee6f31019234e67113173677d18f"
server_key_iv: "b913885865c719550732bfbf5148b900"
server_key_version: "encryptionkey@0"
I20260927 18:39:27.854406 20150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:27.870170 20150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:27.872605 20150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:27.872773 20150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:27.873351 20150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:27.873390 20150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:27.873414 20150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:27.873461 20150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:39:27 dist-test-slave-z47j krb5kdc[19753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534367, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM
I20260927 18:39:27.881604 20150 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:33395
I20260927 18:39:27.881771 20280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:33395 every 8 connection(s)
I20260927 18:39:27.882097 20150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:39:27.884804 19848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:27.873912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:51095 (local address 127.11.251.254:46095)
0927 18:39:27.873994 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:27.873997 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:27.874208 (+ 211us) server_negotiation.cc:407] Connection header received
0927 18:39:27.874772 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:27.874773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:27.874790 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:27.874880 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:27.875551 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.876461 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.878169 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.878519 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.878603 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:27.881693 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:27.881696 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:27.881696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:27.881705 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:27.883314 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:27.883792 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:27.883793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:27.883793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:27.883851 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:27.884107 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:27.884109 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:27.884109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:27.884253 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:27.884324 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:27.884634 (+ 310us) server_negotiation.cc:299] Negotiation successful
0927 18:39:27.884717 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:39:27.885042 20174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:27.873880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:46095 (local address 127.11.251.195:51095)
0927 18:39:27.874055 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:27.874099 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:27.874667 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:27.874970 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:27.874974 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:27.875250 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:27.875469 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:27.875473 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.876585 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.876586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:27.878086 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:27.878089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.878619 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.878619 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:27.878785 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:27.879468 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:27.879478 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:27.881517 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:27.883412 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:27.883417 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:27.883419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:27.883721 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:27.883912 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:27.883913 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:27.883913 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:27.884012 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:27.884339 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:27.884345 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:27.884474 (+ 129us) client_negotiation.cc:769] Sending connection context
0927 18:39:27.884578 (+ 104us) client_negotiation.cc:240] Negotiation successful
0927 18:39:27.884738 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":103,"threads_started":1}
I20260927 18:39:27.885547 20281 heartbeater.cc:347] Connected to a master server at 127.11.251.254:46095
I20260927 18:39:27.885656 20281 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:27.885867 20281 heartbeater.cc:511] Master 127.11.251.254:46095 requested a full tablet report, sending...
I20260927 18:39:27.886333 19797 ts_manager.cc:194] Registered new tserver with Master: 87bc6c3807a64aa2891412273b2eb0fd (127.11.251.195:33395)
I20260927 18:39:27.886762 19797 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:51095
I20260927 18:39:27.891399 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 20150
I20260927 18:39:27.891520 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:39:27.891795 12271 external_mini_cluster.cc:1428] Setting key 18d2c4451b2bb81ecc5b393d1c5dfba5
I20260927 18:39:27.892454 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:27.886754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:46095 (local address 127.11.251.195:53309)
0927 18:39:27.886893 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:27.886905 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:27.886965 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:27.887152 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:27.887153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:27.887364 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:27.887439 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:27.887441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.888087 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.888088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:27.888583 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:27.888584 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.888902 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.888903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:27.888949 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:27.889423 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:27.889428 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:27.889805 (+ 377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:27.891170 (+ 1365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:27.891171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:27.891171 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:27.891416 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:27.891668 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:27.891669 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:27.891670 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:27.891738 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:27.892062 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:27.892063 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:27.892109 (+ 46us) client_negotiation.cc:769] Sending connection context
0927 18:39:27.892211 (+ 102us) client_negotiation.cc:240] Negotiation successful
0927 18:39:27.892327 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260927 18:39:27.892432 19848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:27.886825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:53309 (local address 127.11.251.254:46095)
0927 18:39:27.886911 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:27.886913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:27.886951 (+ 38us) server_negotiation.cc:407] Connection header received
0927 18:39:27.887011 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:27.887012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:27.887023 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:27.887096 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:27.887514 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.888002 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.888652 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.888840 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.888891 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:27.889905 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:27.889906 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:27.889906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:27.889912 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:27.891094 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:27.891516 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:27.891517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:27.891518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:27.891580 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:27.891823 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:27.891824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:27.891824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:27.891984 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:27.892074 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:27.892261 (+ 187us) server_negotiation.cc:299] Negotiation successful
0927 18:39:27.892348 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260927 18:39:27.892868 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 18:39:27 dist-test-slave-z47j krb5kdc[19753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534367, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 18:39:27 dist-test-slave-z47j krb5kdc[19753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:27 dist-test-slave-z47j krb5kdc[19753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
W20260927 18:39:27.919449 19848 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:27.912909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53680 (local address 127.11.251.254:46095)
0927 18:39:27.912998 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:27.913000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:27.913012 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:39:27.913059 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:27.913060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:27.913072 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:27.913133 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:27.914130 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.914726 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.915506 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.915698 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.915755 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:27.919302 (+ 3547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53680: BlockingRecv error: recv got EOF from 127.0.0.1:53680 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 27 18:39:27 dist-test-slave-z47j krb5kdc[19753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534367, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM
I20260927 18:39:27.928385 19848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:27.920300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53696 (local address 127.11.251.254:46095)
0927 18:39:27.920369 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:27.920371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:27.920399 (+ 28us) server_negotiation.cc:407] Connection header received
0927 18:39:27.920466 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:27.920467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:27.920476 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:27.920530 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:27.921229 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.922041 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.922841 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.923057 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.923127 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:27.925743 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:27.925744 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:27.925745 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:27.925752 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:27.927276 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:27.927596 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:27.927597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:27.927597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:27.927652 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:27.927835 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:27.927836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:27.927836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:27.927945 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:27.928026 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:27.928219 (+ 193us) server_negotiation.cc:299] Negotiation successful
0927 18:39:27.928297 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 18:39:27.930605 19798 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:39:27.932762 19798 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:39:27.940249 20305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:27.935584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:42283 (local address 127.0.0.1:42516)
0927 18:39:27.935770 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:27.935825 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:27.935903 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:27.936491 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:27.936492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:27.936494 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:27.936556 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:27.936558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.937931 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.937932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:27.938983 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:27.938985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.939893 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.939894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:27.939975 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:27.939991 (+ 16us) client_negotiation.cc:769] Sending connection context
0927 18:39:27.940064 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 18:39:27.940122 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":55,"threads_started":1}
I20260927 18:39:27.940266 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:27.935891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42516 (local address 127.11.251.193:42283)
0927 18:39:27.936309 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:27.936314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:27.936324 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:39:27.936369 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:27.936371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:27.936387 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:27.936494 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:27.936636 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.937827 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.939062 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.939783 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.939916 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:27.939965 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:27.940094 (+ 129us) server_negotiation.cc:299] Negotiation successful
0927 18:39:27.940146 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":99,"threads_started":1}
I20260927 18:39:27.940588 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:27.936111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37318 (local address 127.11.251.194:43985)
0927 18:39:27.936358 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:27.936362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:27.936372 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:39:27.936410 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:27.936413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:27.936428 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:27.936507 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:27.936707 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.937436 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.938946 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.939783 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.939883 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:27.939956 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:27.940297 (+ 341us) server_negotiation.cc:299] Negotiation successful
0927 18:39:27.940460 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":111,"threads_started":1}
I20260927 18:39:27.940891 20303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:27.935584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:43985 (local address 127.0.0.1:37318)
0927 18:39:27.935851 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:27.935863 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:27.935926 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:27.936587 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:27.936589 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:27.936591 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:27.936631 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:27.936633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.937529 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.937530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:27.938849 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:27.938851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.939992 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.939993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:27.940067 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:27.940072 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:39:27.940124 (+ 52us) client_negotiation.cc:240] Negotiation successful
0927 18:39:27.940180 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":3,"spinlock_wait_cycles":768,"thread_start_us":119,"threads_started":1}
I20260927 18:39:27.941144 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:27.935874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46000 (local address 127.11.251.195:33395)
0927 18:39:27.936125 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:27.936130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:27.936181 (+ 51us) server_negotiation.cc:407] Connection header received
0927 18:39:27.936255 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:27.936258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:27.936274 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:27.936362 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:27.936620 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.937837 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.939834 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.940587 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.940691 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:27.940750 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:27.940971 (+ 221us) server_negotiation.cc:299] Negotiation successful
0927 18:39:27.941030 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":107,"threads_started":1}
I20260927 18:39:27.941064 20304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:27.935588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:33395 (local address 127.0.0.1:46000)
0927 18:39:27.936088 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:27.936096 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:27.936176 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:27.936463 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:27.936466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:27.936469 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:27.936547 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:27.936550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.939055 (+ 2505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.939055 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:27.939739 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:27.939740 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:27.940688 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:27.940689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:27.940777 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:27.940782 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:39:27.940848 (+ 66us) client_negotiation.cc:240] Negotiation successful
0927 18:39:27.940882 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":97,"threads_started":1}
I20260927 18:39:27.941680 19935 tablet_service.cc:1467] Processing CreateTablet for tablet 83e243fbee1041d5b9a991deb4865f88 (DEFAULT_TABLE table=test-table [id=c6454d30ec6544e38c4bbd0ada398bb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:27.942193 19935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83e243fbee1041d5b9a991deb4865f88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:27.942340 20215 tablet_service.cc:1467] Processing CreateTablet for tablet 83e243fbee1041d5b9a991deb4865f88 (DEFAULT_TABLE table=test-table [id=c6454d30ec6544e38c4bbd0ada398bb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:27.942447 20075 tablet_service.cc:1467] Processing CreateTablet for tablet 83e243fbee1041d5b9a991deb4865f88 (DEFAULT_TABLE table=test-table [id=c6454d30ec6544e38c4bbd0ada398bb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:27.942680 20215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83e243fbee1041d5b9a991deb4865f88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:27.942804 20075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83e243fbee1041d5b9a991deb4865f88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:27.946022 20309 tablet_bootstrap.cc:492] T 83e243fbee1041d5b9a991deb4865f88 P 87bc6c3807a64aa2891412273b2eb0fd: Bootstrap starting.
I20260927 18:39:27.946877 20309 tablet_bootstrap.cc:654] T 83e243fbee1041d5b9a991deb4865f88 P 87bc6c3807a64aa2891412273b2eb0fd: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:27.947176 20311 tablet_bootstrap.cc:492] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4: Bootstrap starting.
I20260927 18:39:27.947252 20309 log.cc:824] T 83e243fbee1041d5b9a991deb4865f88 P 87bc6c3807a64aa2891412273b2eb0fd: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:27.947305 20310 tablet_bootstrap.cc:492] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e: Bootstrap starting.
I20260927 18:39:27.947899 20311 tablet_bootstrap.cc:654] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:27.947927 20310 tablet_bootstrap.cc:654] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:27.947999 20309 tablet_bootstrap.cc:492] T 83e243fbee1041d5b9a991deb4865f88 P 87bc6c3807a64aa2891412273b2eb0fd: No bootstrap required, opened a new log
I20260927 18:39:27.948065 20309 ts_tablet_manager.cc:1397] T 83e243fbee1041d5b9a991deb4865f88 P 87bc6c3807a64aa2891412273b2eb0fd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:39:27.948264 20310 log.cc:824] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:27.948271 20311 log.cc:824] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:27.949110 20310 tablet_bootstrap.cc:492] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e: No bootstrap required, opened a new log
I20260927 18:39:27.949204 20310 ts_tablet_manager.cc:1397] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:39:27.949321 20311 tablet_bootstrap.cc:492] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4: No bootstrap required, opened a new log
I20260927 18:39:27.949393 20311 ts_tablet_manager.cc:1397] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:39:27.950179 20309 raft_consensus.cc:348] T 83e243fbee1041d5b9a991deb4865f88 P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:27.950378 20309 raft_consensus.cc:374] T 83e243fbee1041d5b9a991deb4865f88 P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:27.950424 20309 raft_consensus.cc:729] T 83e243fbee1041d5b9a991deb4865f88 P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87bc6c3807a64aa2891412273b2eb0fd, State: Initialized, Role: FOLLOWER
I20260927 18:39:27.950552 20309 consensus_queue.cc:260] T 83e243fbee1041d5b9a991deb4865f88 P 87bc6c3807a64aa2891412273b2eb0fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:27.950875 20309 ts_tablet_manager.cc:1428] T 83e243fbee1041d5b9a991deb4865f88 P 87bc6c3807a64aa2891412273b2eb0fd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:39:27.951006 20281 heartbeater.cc:503] Master 127.11.251.254:46095 was elected leader, sending a full tablet report...
I20260927 18:39:27.951710 20311 raft_consensus.cc:348] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:27.951710 20310 raft_consensus.cc:348] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:27.951889 20311 raft_consensus.cc:374] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:27.951889 20310 raft_consensus.cc:374] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:27.951934 20311 raft_consensus.cc:729] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c73d576dc1304b8181c35475c71a35e4, State: Initialized, Role: FOLLOWER
I20260927 18:39:27.951934 20310 raft_consensus.cc:729] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eb34beb910b4457a618e130cb7c8b5e, State: Initialized, Role: FOLLOWER
I20260927 18:39:27.952059 20310 consensus_queue.cc:260] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:27.952059 20311 consensus_queue.cc:260] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:27.952355 20311 ts_tablet_manager.cc:1428] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:39:27.952354 20310 ts_tablet_manager.cc:1428] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260927 18:39:27.952452 20141 heartbeater.cc:503] Master 127.11.251.254:46095 was elected leader, sending a full tablet report...
I20260927 18:39:27.952518 20001 heartbeater.cc:503] Master 127.11.251.254:46095 was elected leader, sending a full tablet report...
W20260927 18:39:27.963964 20142 tablet.cc:2369] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:39:28.008095 20316 raft_consensus.cc:482] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:39:28.008351 20316 raft_consensus.cc:504] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.008857 20316 leader_election.cc:288] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c73d576dc1304b8181c35475c71a35e4 (127.11.251.194:43985), 87bc6c3807a64aa2891412273b2eb0fd (127.11.251.195:33395)
I20260927 18:39:28.011971 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.009222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:47077 (local address 127.11.251.194:43985)
0927 18:39:28.009295 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:28.009297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:28.009306 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:39:28.009337 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:28.009338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:28.009348 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:28.009426 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:28.009640 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.010164 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.011192 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.011581 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.011671 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.011683 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:28.011856 (+ 173us) server_negotiation.cc:299] Negotiation successful
0927 18:39:28.011894 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 18:39:28.011989 19894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.009093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:43985 (local address 127.11.251.193:47077)
0927 18:39:28.009180 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:28.009197 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:28.009259 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:28.009432 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:28.009433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:28.009435 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:28.009532 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:28.009534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.010276 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.010277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:28.011079 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:28.011081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.011669 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.011675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:28.011769 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.011773 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:39:28.011837 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 18:39:28.011888 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":1}
I20260927 18:39:28.012082 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.009221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:57199 (local address 127.11.251.195:33395)
0927 18:39:28.009282 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:28.009285 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:28.009437 (+ 152us) server_negotiation.cc:407] Connection header received
0927 18:39:28.009509 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:28.009510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:28.009521 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:28.009568 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:28.009785 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.010293 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.011258 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.011683 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.011770 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.011787 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:28.011943 (+ 156us) server_negotiation.cc:299] Negotiation successful
0927 18:39:28.011984 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 18:39:28.012182 20318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.009093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:33395 (local address 127.11.251.193:57199)
0927 18:39:28.009342 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:28.009352 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:28.009445 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:28.009611 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:28.009611 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:28.009613 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:28.009707 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:28.009709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.010383 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.010384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:28.011160 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:28.011163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.011771 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.011772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:28.011856 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.011860 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:39:28.011937 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 18:39:28.011981 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":24,"thread_start_us":142,"threads_started":1}
I20260927 18:39:28.012419 20095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83e243fbee1041d5b9a991deb4865f88" candidate_uuid: "9eb34beb910b4457a618e130cb7c8b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c73d576dc1304b8181c35475c71a35e4" is_pre_election: true
I20260927 18:39:28.012648 20095 raft_consensus.cc:2454] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9eb34beb910b4457a618e130cb7c8b5e in term 0.
I20260927 18:39:28.012627 20235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83e243fbee1041d5b9a991deb4865f88" candidate_uuid: "9eb34beb910b4457a618e130cb7c8b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87bc6c3807a64aa2891412273b2eb0fd" is_pre_election: true
I20260927 18:39:28.012801 20235 raft_consensus.cc:2454] T 83e243fbee1041d5b9a991deb4865f88 P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9eb34beb910b4457a618e130cb7c8b5e in term 0.
I20260927 18:39:28.012926 19888 leader_election.cc:302] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9eb34beb910b4457a618e130cb7c8b5e, c73d576dc1304b8181c35475c71a35e4; no voters:
I20260927 18:39:28.013062 20316 raft_consensus.cc:2781] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:39:28.013118 20316 raft_consensus.cc:482] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:39:28.013144 20316 raft_consensus.cc:3037] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:28.014364 20316 raft_consensus.cc:504] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.014580 20316 leader_election.cc:288] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e [CANDIDATE]: Term 1 election: Requested vote from peers c73d576dc1304b8181c35475c71a35e4 (127.11.251.194:43985), 87bc6c3807a64aa2891412273b2eb0fd (127.11.251.195:33395)
I20260927 18:39:28.014744 20095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83e243fbee1041d5b9a991deb4865f88" candidate_uuid: "9eb34beb910b4457a618e130cb7c8b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c73d576dc1304b8181c35475c71a35e4"
I20260927 18:39:28.014810 20235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83e243fbee1041d5b9a991deb4865f88" candidate_uuid: "9eb34beb910b4457a618e130cb7c8b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87bc6c3807a64aa2891412273b2eb0fd"
I20260927 18:39:28.014854 20095 raft_consensus.cc:3037] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:28.014884 20235 raft_consensus.cc:3037] T 83e243fbee1041d5b9a991deb4865f88 P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:28.015880 20235 raft_consensus.cc:2454] T 83e243fbee1041d5b9a991deb4865f88 P 87bc6c3807a64aa2891412273b2eb0fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9eb34beb910b4457a618e130cb7c8b5e in term 1.
I20260927 18:39:28.015882 20095 raft_consensus.cc:2454] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9eb34beb910b4457a618e130cb7c8b5e in term 1.
I20260927 18:39:28.016047 19888 leader_election.cc:302] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9eb34beb910b4457a618e130cb7c8b5e, c73d576dc1304b8181c35475c71a35e4; no voters:
I20260927 18:39:28.016156 20316 raft_consensus.cc:2781] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:28.016330 20316 raft_consensus.cc:686] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e [term 1 LEADER]: Becoming Leader. State: Replica: 9eb34beb910b4457a618e130cb7c8b5e, State: Running, Role: LEADER
I20260927 18:39:28.016449 20316 consensus_queue.cc:237] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.017620 19796 catalog_manager.cc:5495] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9eb34beb910b4457a618e130cb7c8b5e (127.11.251.193). New cstate: current_term: 1 leader_uuid: "9eb34beb910b4457a618e130cb7c8b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } health_report { overall_health: UNKNOWN } } }
I20260927 18:39:28.032133 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.029251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42518 (local address 127.11.251.193:42283)
0927 18:39:28.029314 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:28.029315 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:28.029323 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:39:28.029392 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:28.029393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:28.029411 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:28.029475 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:39:28.029707 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.030258 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.031277 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.031467 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.031525 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.031678 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:39:28.031801 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:39:28.031865 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:28.032038 (+ 173us) server_negotiation.cc:299] Negotiation successful
0927 18:39:28.032064 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 18:39:28.034258 20095 raft_consensus.cc:1261] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4 [term 1 FOLLOWER]: Refusing update from remote peer 9eb34beb910b4457a618e130cb7c8b5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:28.034258 20235 raft_consensus.cc:1261] T 83e243fbee1041d5b9a991deb4865f88 P 87bc6c3807a64aa2891412273b2eb0fd [term 1 FOLLOWER]: Refusing update from remote peer 9eb34beb910b4457a618e130cb7c8b5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:28.034734 20319 consensus_queue.cc:1035] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:28.034907 20316 consensus_queue.cc:1035] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:28.037407 20322 mvcc.cc:205] Tried to move back new op lower bound from 7334028771464933376 to 7334028771396210688. Current Snapshot: MvccSnapshot[applied={T|T < 7334028771464933376}]
I20260927 18:39:28.038532 20324 mvcc.cc:205] Tried to move back new op lower bound from 7334028771464933376 to 7334028771396210688. Current Snapshot: MvccSnapshot[applied={T|T < 7334028771464933376 or (T in {7334028771464933376})}]
I20260927 18:39:28.040467 19796 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:28.040584 19796 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:28.041469 19796 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad: Sending DeleteTablet for 3 replicas of tablet 83e243fbee1041d5b9a991deb4865f88
I20260927 18:39:28.041914 19935 tablet_service.cc:1514] Processing DeleteTablet for tablet 83e243fbee1041d5b9a991deb4865f88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:28 UTC) from {username='oryx', principal='oryx/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:42516
I20260927 18:39:28.042060 19796 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:39:28.042320 19796 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:39:28.042361 20215 tablet_service.cc:1514] Processing DeleteTablet for tablet 83e243fbee1041d5b9a991deb4865f88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:28 UTC) from {username='oryx', principal='oryx/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:46000
I20260927 18:39:28.042721 20333 tablet_replica.cc:330] T 83e243fbee1041d5b9a991deb4865f88 P 87bc6c3807a64aa2891412273b2eb0fd: stopping tablet replica
I20260927 18:39:28.042708 20075 tablet_service.cc:1514] Processing DeleteTablet for tablet 83e243fbee1041d5b9a991deb4865f88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:28 UTC) from {username='oryx', principal='oryx/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:37318
I20260927 18:39:28.042836 20333 raft_consensus.cc:2229] T 83e243fbee1041d5b9a991deb4865f88 P 87bc6c3807a64aa2891412273b2eb0fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:28.042972 20333 raft_consensus.cc:2258] T 83e243fbee1041d5b9a991deb4865f88 P 87bc6c3807a64aa2891412273b2eb0fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:28.043586 20334 tablet_replica.cc:330] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4: stopping tablet replica
I20260927 18:39:28.043644 20333 ts_tablet_manager.cc:1905] T 83e243fbee1041d5b9a991deb4865f88 P 87bc6c3807a64aa2891412273b2eb0fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:28.043708 20334 raft_consensus.cc:2229] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:28.043998 20334 raft_consensus.cc:2258] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:28.044628 20215 tablet_service.cc:1467] Processing CreateTablet for tablet ae248dc03b584d4b8e45bc933a2d619b (DEFAULT_TABLE table=test-table [id=92b4d69b1d324fbaaa530a25f2fb6970]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:28.044781 20215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae248dc03b584d4b8e45bc933a2d619b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:28.045948 20332 tablet_replica.cc:330] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e: stopping tablet replica
I20260927 18:39:28.046063 20332 raft_consensus.cc:2229] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:39:28.046218 20309 tablet_bootstrap.cc:492] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd: Bootstrap starting.
I20260927 18:39:28.046350 19935 tablet_service.cc:1467] Processing CreateTablet for tablet ae248dc03b584d4b8e45bc933a2d619b (DEFAULT_TABLE table=test-table [id=92b4d69b1d324fbaaa530a25f2fb6970]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:28.046519 20333 ts_tablet_manager.cc:1918] T 83e243fbee1041d5b9a991deb4865f88 P 87bc6c3807a64aa2891412273b2eb0fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:39:28.046609 20333 log.cc:1196] T 83e243fbee1041d5b9a991deb4865f88 P 87bc6c3807a64aa2891412273b2eb0fd: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/83e243fbee1041d5b9a991deb4865f88
I20260927 18:39:28.046695 20309 tablet_bootstrap.cc:654] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:28.046746 20075 tablet_service.cc:1467] Processing CreateTablet for tablet ae248dc03b584d4b8e45bc933a2d619b (DEFAULT_TABLE table=test-table [id=92b4d69b1d324fbaaa530a25f2fb6970]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:28.046877 20075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae248dc03b584d4b8e45bc933a2d619b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:28.046900 20333 ts_tablet_manager.cc:1939] T 83e243fbee1041d5b9a991deb4865f88 P 87bc6c3807a64aa2891412273b2eb0fd: Deleting consensus metadata
I20260927 18:39:28.047286 19935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae248dc03b584d4b8e45bc933a2d619b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:28.048440 20311 tablet_bootstrap.cc:492] T ae248dc03b584d4b8e45bc933a2d619b P c73d576dc1304b8181c35475c71a35e4: Bootstrap starting.
I20260927 18:39:28.048614 19783 catalog_manager.cc:4841] TS 87bc6c3807a64aa2891412273b2eb0fd (127.11.251.195:33395): tablet 83e243fbee1041d5b9a991deb4865f88 (table test-table [id=c6454d30ec6544e38c4bbd0ada398bb9]) successfully deleted
I20260927 18:39:28.048909 20310 tablet_bootstrap.cc:492] T ae248dc03b584d4b8e45bc933a2d619b P 9eb34beb910b4457a618e130cb7c8b5e: Bootstrap starting.
I20260927 18:39:28.048945 20311 tablet_bootstrap.cc:654] T ae248dc03b584d4b8e45bc933a2d619b P c73d576dc1304b8181c35475c71a35e4: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:28.049166 20334 ts_tablet_manager.cc:1905] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:28.049362 20310 tablet_bootstrap.cc:654] T ae248dc03b584d4b8e45bc933a2d619b P 9eb34beb910b4457a618e130cb7c8b5e: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:28.050102 20310 tablet_bootstrap.cc:492] T ae248dc03b584d4b8e45bc933a2d619b P 9eb34beb910b4457a618e130cb7c8b5e: No bootstrap required, opened a new log
I20260927 18:39:28.050168 20310 ts_tablet_manager.cc:1397] T ae248dc03b584d4b8e45bc933a2d619b P 9eb34beb910b4457a618e130cb7c8b5e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260927 18:39:28.050292 20332 raft_consensus.cc:2258] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:28.050408 20310 raft_consensus.cc:348] T ae248dc03b584d4b8e45bc933a2d619b P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.050484 20310 raft_consensus.cc:374] T ae248dc03b584d4b8e45bc933a2d619b P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:28.050505 20310 raft_consensus.cc:729] T ae248dc03b584d4b8e45bc933a2d619b P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eb34beb910b4457a618e130cb7c8b5e, State: Initialized, Role: FOLLOWER
I20260927 18:39:28.050556 20310 consensus_queue.cc:260] T ae248dc03b584d4b8e45bc933a2d619b P 9eb34beb910b4457a618e130cb7c8b5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.050652 20310 ts_tablet_manager.cc:1428] T ae248dc03b584d4b8e45bc933a2d619b P 9eb34beb910b4457a618e130cb7c8b5e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:28.050787 20332 ts_tablet_manager.cc:1905] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:28.051096 20309 tablet_bootstrap.cc:492] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd: No bootstrap required, opened a new log
I20260927 18:39:28.051180 20309 ts_tablet_manager.cc:1397] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260927 18:39:28.051322 20334 ts_tablet_manager.cc:1918] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:39:28.051352 20309 raft_consensus.cc:348] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.051440 20309 raft_consensus.cc:374] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:28.051472 20309 raft_consensus.cc:729] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87bc6c3807a64aa2891412273b2eb0fd, State: Initialized, Role: FOLLOWER
I20260927 18:39:28.051429 20334 log.cc:1196] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/83e243fbee1041d5b9a991deb4865f88
I20260927 18:39:28.051529 20309 consensus_queue.cc:260] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.051781 20309 ts_tablet_manager.cc:1428] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:28.052609 20334 ts_tablet_manager.cc:1939] T 83e243fbee1041d5b9a991deb4865f88 P c73d576dc1304b8181c35475c71a35e4: Deleting consensus metadata
I20260927 18:39:28.053071 19784 catalog_manager.cc:4841] TS c73d576dc1304b8181c35475c71a35e4 (127.11.251.194:43985): tablet 83e243fbee1041d5b9a991deb4865f88 (table test-table [id=c6454d30ec6544e38c4bbd0ada398bb9]) successfully deleted
I20260927 18:39:28.053457 20332 ts_tablet_manager.cc:1918] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:39:28.053532 20332 log.cc:1196] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-0/wal/wals/83e243fbee1041d5b9a991deb4865f88
I20260927 18:39:28.053706 20311 tablet_bootstrap.cc:492] T ae248dc03b584d4b8e45bc933a2d619b P c73d576dc1304b8181c35475c71a35e4: No bootstrap required, opened a new log
I20260927 18:39:28.053740 20311 ts_tablet_manager.cc:1397] T ae248dc03b584d4b8e45bc933a2d619b P c73d576dc1304b8181c35475c71a35e4: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260927 18:39:28.053864 20311 raft_consensus.cc:348] T ae248dc03b584d4b8e45bc933a2d619b P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.053851 20332 ts_tablet_manager.cc:1939] T 83e243fbee1041d5b9a991deb4865f88 P 9eb34beb910b4457a618e130cb7c8b5e: Deleting consensus metadata
I20260927 18:39:28.053912 20311 raft_consensus.cc:374] T ae248dc03b584d4b8e45bc933a2d619b P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:28.053925 20311 raft_consensus.cc:729] T ae248dc03b584d4b8e45bc933a2d619b P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c73d576dc1304b8181c35475c71a35e4, State: Initialized, Role: FOLLOWER
I20260927 18:39:28.053988 20311 consensus_queue.cc:260] T ae248dc03b584d4b8e45bc933a2d619b P c73d576dc1304b8181c35475c71a35e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.054076 20311 ts_tablet_manager.cc:1428] T ae248dc03b584d4b8e45bc933a2d619b P c73d576dc1304b8181c35475c71a35e4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:28.054384 19785 catalog_manager.cc:4841] TS 9eb34beb910b4457a618e130cb7c8b5e (127.11.251.193:42283): tablet 83e243fbee1041d5b9a991deb4865f88 (table test-table [id=c6454d30ec6544e38c4bbd0ada398bb9]) successfully deleted
W20260927 18:39:28.132738 20282 tablet.cc:2369] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:39:28.204419 20315 raft_consensus.cc:482] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:39:28.204550 20315 raft_consensus.cc:504] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.205034 20315 leader_election.cc:288] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9eb34beb910b4457a618e130cb7c8b5e (127.11.251.193:42283), c73d576dc1304b8181c35475c71a35e4 (127.11.251.194:43985)
I20260927 18:39:28.209263 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.205398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:35951 (local address 127.11.251.193:42283)
0927 18:39:28.205460 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:28.205462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:28.205475 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:39:28.205512 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:28.205513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:28.205533 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:28.205678 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:28.205895 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.206711 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.208321 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.208883 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.208962 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.208974 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:28.209157 (+ 183us) server_negotiation.cc:299] Negotiation successful
0927 18:39:28.209198 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 18:39:28.209262 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.205419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:39809 (local address 127.11.251.194:43985)
0927 18:39:28.205469 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:28.205471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:28.205571 (+ 100us) server_negotiation.cc:407] Connection header received
0927 18:39:28.205721 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:28.205723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:28.205734 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:28.205796 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:28.205958 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.206712 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.208318 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.208883 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.208962 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.208974 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:28.209136 (+ 162us) server_negotiation.cc:299] Negotiation successful
0927 18:39:28.209184 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:39:28.209275 20174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.205289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:42283 (local address 127.11.251.195:35951)
0927 18:39:28.205361 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:28.205376 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:28.205436 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:28.205689 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:28.205690 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:28.205692 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:28.205803 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:28.205806 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.206801 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.206802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:28.208233 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:28.208235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.208974 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.208975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:28.209065 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.209068 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:39:28.209119 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 18:39:28.209171 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":90}
I20260927 18:39:28.209290 20338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.205320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:43985 (local address 127.11.251.195:39809)
0927 18:39:28.205502 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:28.205511 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:28.205568 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:28.205806 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:28.205806 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:28.205808 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:28.205892 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:28.205893 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.206801 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.206802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:28.208225 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:28.208229 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.208992 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.208993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:28.209055 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.209059 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:39:28.209127 (+ 68us) client_negotiation.cc:240] Negotiation successful
0927 18:39:28.209170 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":101,"thread_start_us":105,"threads_started":1}
I20260927 18:39:28.209661 19955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae248dc03b584d4b8e45bc933a2d619b" candidate_uuid: "87bc6c3807a64aa2891412273b2eb0fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9eb34beb910b4457a618e130cb7c8b5e" is_pre_election: true
I20260927 18:39:28.209823 19955 raft_consensus.cc:2454] T ae248dc03b584d4b8e45bc933a2d619b P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87bc6c3807a64aa2891412273b2eb0fd in term 0.
I20260927 18:39:28.209894 20095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae248dc03b584d4b8e45bc933a2d619b" candidate_uuid: "87bc6c3807a64aa2891412273b2eb0fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c73d576dc1304b8181c35475c71a35e4" is_pre_election: true
I20260927 18:39:28.210019 20095 raft_consensus.cc:2454] T ae248dc03b584d4b8e45bc933a2d619b P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87bc6c3807a64aa2891412273b2eb0fd in term 0.
I20260927 18:39:28.210013 20169 leader_election.cc:302] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87bc6c3807a64aa2891412273b2eb0fd, 9eb34beb910b4457a618e130cb7c8b5e; no voters:
I20260927 18:39:28.210153 20315 raft_consensus.cc:2781] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:39:28.210191 20315 raft_consensus.cc:482] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:39:28.210218 20315 raft_consensus.cc:3037] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:28.211086 20315 raft_consensus.cc:504] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.211251 20315 leader_election.cc:288] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd [CANDIDATE]: Term 1 election: Requested vote from peers 9eb34beb910b4457a618e130cb7c8b5e (127.11.251.193:42283), c73d576dc1304b8181c35475c71a35e4 (127.11.251.194:43985)
I20260927 18:39:28.211400 20095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae248dc03b584d4b8e45bc933a2d619b" candidate_uuid: "87bc6c3807a64aa2891412273b2eb0fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c73d576dc1304b8181c35475c71a35e4"
I20260927 18:39:28.211479 20095 raft_consensus.cc:3037] T ae248dc03b584d4b8e45bc933a2d619b P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:28.212229 20095 raft_consensus.cc:2454] T ae248dc03b584d4b8e45bc933a2d619b P c73d576dc1304b8181c35475c71a35e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87bc6c3807a64aa2891412273b2eb0fd in term 1.
I20260927 18:39:28.212352 19955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae248dc03b584d4b8e45bc933a2d619b" candidate_uuid: "87bc6c3807a64aa2891412273b2eb0fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9eb34beb910b4457a618e130cb7c8b5e"
I20260927 18:39:28.212384 20168 leader_election.cc:302] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87bc6c3807a64aa2891412273b2eb0fd, c73d576dc1304b8181c35475c71a35e4; no voters:
I20260927 18:39:28.212435 19955 raft_consensus.cc:3037] T ae248dc03b584d4b8e45bc933a2d619b P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:28.212477 20315 raft_consensus.cc:2781] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:28.212641 20315 raft_consensus.cc:686] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd [term 1 LEADER]: Becoming Leader. State: Replica: 87bc6c3807a64aa2891412273b2eb0fd, State: Running, Role: LEADER
I20260927 18:39:28.212725 20315 consensus_queue.cc:237] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.213357 19955 raft_consensus.cc:2454] T ae248dc03b584d4b8e45bc933a2d619b P 9eb34beb910b4457a618e130cb7c8b5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87bc6c3807a64aa2891412273b2eb0fd in term 1.
I20260927 18:39:28.213560 19798 catalog_manager.cc:5495] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 87bc6c3807a64aa2891412273b2eb0fd (127.11.251.195). New cstate: current_term: 1 leader_uuid: "87bc6c3807a64aa2891412273b2eb0fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } health_report { overall_health: HEALTHY } } }
Sep 27 18:39:28 dist-test-slave-z47j krb5kdc[19753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534367, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.254@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM
I20260927 18:39:28.258476 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.250699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:46095 (local address 127.0.0.1:53706)
0927 18:39:28.251062 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:28.251084 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:28.251188 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:28.251377 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:28.251378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:28.251778 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:28.251844 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:28.251847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.252676 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.252677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:28.253337 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:28.253340 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.253771 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.253772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:28.253823 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.254371 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:28.254376 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:28.255894 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:28.257377 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:28.257384 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:28.257386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:28.257632 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:28.257818 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:28.257819 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:28.257819 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:28.257879 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:28.258097 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:28.258104 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:28.258170 (+ 66us) client_negotiation.cc:769] Sending connection context
0927 18:39:28.258226 (+ 56us) client_negotiation.cc:240] Negotiation successful
0927 18:39:28.258367 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":1,"thread_start_us":218,"threads_started":1}
I20260927 18:39:28.258491 19848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.250913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53706 (local address 127.11.251.254:46095)
0927 18:39:28.251031 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:28.251034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:28.251164 (+ 130us) server_negotiation.cc:407] Connection header received
0927 18:39:28.251285 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:28.251287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:28.251307 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:28.251374 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:28.251940 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.252580 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.253432 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.253691 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.253772 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.255977 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:28.255979 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:28.255980 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:28.255985 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:28.257306 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:28.257704 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:28.257705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:28.257706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:28.257760 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:28.257932 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:28.257933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:28.257933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:28.258038 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:28.258087 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:28.258269 (+ 182us) server_negotiation.cc:299] Negotiation successful
0927 18:39:28.258416 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":76,"server-negotiator.queue_time_us":77}
I20260927 18:39:28.260682 19798 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:53706:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 18:39:28.260974 19798 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:39:28.262669 20215 tablet_service.cc:1467] Processing CreateTablet for tablet 30d5b069f43844b8979287e078c63d21 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e7ff6fedba834434b2f5be7e56ab31b8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:39:28.262653 19935 tablet_service.cc:1467] Processing CreateTablet for tablet 30d5b069f43844b8979287e078c63d21 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e7ff6fedba834434b2f5be7e56ab31b8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:39:28.262703 20075 tablet_service.cc:1467] Processing CreateTablet for tablet 30d5b069f43844b8979287e078c63d21 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e7ff6fedba834434b2f5be7e56ab31b8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:39:28.262849 19935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30d5b069f43844b8979287e078c63d21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:28.262848 20215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30d5b069f43844b8979287e078c63d21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:28.262883 20075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30d5b069f43844b8979287e078c63d21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:28.264341 20311 tablet_bootstrap.cc:492] T 30d5b069f43844b8979287e078c63d21 P c73d576dc1304b8181c35475c71a35e4: Bootstrap starting.
I20260927 18:39:28.264569 20309 tablet_bootstrap.cc:492] T 30d5b069f43844b8979287e078c63d21 P 87bc6c3807a64aa2891412273b2eb0fd: Bootstrap starting.
I20260927 18:39:28.264842 20311 tablet_bootstrap.cc:654] T 30d5b069f43844b8979287e078c63d21 P c73d576dc1304b8181c35475c71a35e4: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:28.265048 20309 tablet_bootstrap.cc:654] T 30d5b069f43844b8979287e078c63d21 P 87bc6c3807a64aa2891412273b2eb0fd: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:28.265241 20310 tablet_bootstrap.cc:492] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e: Bootstrap starting.
I20260927 18:39:28.265673 20310 tablet_bootstrap.cc:654] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:28.265951 20309 tablet_bootstrap.cc:492] T 30d5b069f43844b8979287e078c63d21 P 87bc6c3807a64aa2891412273b2eb0fd: No bootstrap required, opened a new log
I20260927 18:39:28.266021 20309 ts_tablet_manager.cc:1397] T 30d5b069f43844b8979287e078c63d21 P 87bc6c3807a64aa2891412273b2eb0fd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:28.266111 20311 tablet_bootstrap.cc:492] T 30d5b069f43844b8979287e078c63d21 P c73d576dc1304b8181c35475c71a35e4: No bootstrap required, opened a new log
I20260927 18:39:28.266165 20311 ts_tablet_manager.cc:1397] T 30d5b069f43844b8979287e078c63d21 P c73d576dc1304b8181c35475c71a35e4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:39:28.266342 20311 raft_consensus.cc:348] T 30d5b069f43844b8979287e078c63d21 P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.266408 20311 raft_consensus.cc:374] T 30d5b069f43844b8979287e078c63d21 P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:28.266422 20311 raft_consensus.cc:729] T 30d5b069f43844b8979287e078c63d21 P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c73d576dc1304b8181c35475c71a35e4, State: Initialized, Role: FOLLOWER
I20260927 18:39:28.266398 20309 raft_consensus.cc:348] T 30d5b069f43844b8979287e078c63d21 P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.266481 20311 consensus_queue.cc:260] T 30d5b069f43844b8979287e078c63d21 P c73d576dc1304b8181c35475c71a35e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.266505 20309 raft_consensus.cc:374] T 30d5b069f43844b8979287e078c63d21 P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:28.266530 20309 raft_consensus.cc:729] T 30d5b069f43844b8979287e078c63d21 P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87bc6c3807a64aa2891412273b2eb0fd, State: Initialized, Role: FOLLOWER
I20260927 18:39:28.266580 20309 consensus_queue.cc:260] T 30d5b069f43844b8979287e078c63d21 P 87bc6c3807a64aa2891412273b2eb0fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.266597 20311 ts_tablet_manager.cc:1428] T 30d5b069f43844b8979287e078c63d21 P c73d576dc1304b8181c35475c71a35e4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:28.266677 20309 ts_tablet_manager.cc:1428] T 30d5b069f43844b8979287e078c63d21 P 87bc6c3807a64aa2891412273b2eb0fd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:28.266646 20317 tablet_replica.cc:439] T 30d5b069f43844b8979287e078c63d21 P c73d576dc1304b8181c35475c71a35e4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } } }
I20260927 18:39:28.266705 20315 tablet_replica.cc:439] T 30d5b069f43844b8979287e078c63d21 P 87bc6c3807a64aa2891412273b2eb0fd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } } }
I20260927 18:39:28.266747 20317 tablet_replica.cc:442] T 30d5b069f43844b8979287e078c63d21 P c73d576dc1304b8181c35475c71a35e4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:39:28.266798 20315 tablet_replica.cc:442] T 30d5b069f43844b8979287e078c63d21 P 87bc6c3807a64aa2891412273b2eb0fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:39:28.266812 20310 tablet_bootstrap.cc:492] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e: No bootstrap required, opened a new log
I20260927 18:39:28.266852 20310 ts_tablet_manager.cc:1397] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:39:28.266991 20310 raft_consensus.cc:348] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.267073 20310 raft_consensus.cc:374] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:28.267107 20310 raft_consensus.cc:729] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eb34beb910b4457a618e130cb7c8b5e, State: Initialized, Role: FOLLOWER
I20260927 18:39:28.267149 20310 consensus_queue.cc:260] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.267238 20310 ts_tablet_manager.cc:1428] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:28.267261 20319 tablet_replica.cc:439] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } } }
I20260927 18:39:28.267364 20319 tablet_replica.cc:442] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e: This TxnStatusTablet replica's current role is: FOLLOWER
W20260927 18:39:28.299500 20002 tablet.cc:2369] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:39:28.446231 20319 raft_consensus.cc:482] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:39:28.446403 20319 raft_consensus.cc:504] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.446645 20319 leader_election.cc:288] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c73d576dc1304b8181c35475c71a35e4 (127.11.251.194:43985), 87bc6c3807a64aa2891412273b2eb0fd (127.11.251.195:33395)
I20260927 18:39:28.446877 20095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d5b069f43844b8979287e078c63d21" candidate_uuid: "9eb34beb910b4457a618e130cb7c8b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c73d576dc1304b8181c35475c71a35e4" is_pre_election: true
I20260927 18:39:28.446983 20095 raft_consensus.cc:2454] T 30d5b069f43844b8979287e078c63d21 P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9eb34beb910b4457a618e130cb7c8b5e in term 0.
I20260927 18:39:28.446887 20235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d5b069f43844b8979287e078c63d21" candidate_uuid: "9eb34beb910b4457a618e130cb7c8b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87bc6c3807a64aa2891412273b2eb0fd" is_pre_election: true
I20260927 18:39:28.447052 20235 raft_consensus.cc:2454] T 30d5b069f43844b8979287e078c63d21 P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9eb34beb910b4457a618e130cb7c8b5e in term 0.
I20260927 18:39:28.447192 19888 leader_election.cc:302] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9eb34beb910b4457a618e130cb7c8b5e, c73d576dc1304b8181c35475c71a35e4; no voters:
I20260927 18:39:28.447357 20319 raft_consensus.cc:2781] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:39:28.447403 20319 raft_consensus.cc:482] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:39:28.447435 20319 raft_consensus.cc:3037] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:28.448408 20319 raft_consensus.cc:504] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.448568 20319 leader_election.cc:288] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e [CANDIDATE]: Term 1 election: Requested vote from peers c73d576dc1304b8181c35475c71a35e4 (127.11.251.194:43985), 87bc6c3807a64aa2891412273b2eb0fd (127.11.251.195:33395)
I20260927 18:39:28.448731 20235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d5b069f43844b8979287e078c63d21" candidate_uuid: "9eb34beb910b4457a618e130cb7c8b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87bc6c3807a64aa2891412273b2eb0fd"
I20260927 18:39:28.448822 20235 raft_consensus.cc:3037] T 30d5b069f43844b8979287e078c63d21 P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:28.448774 20095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d5b069f43844b8979287e078c63d21" candidate_uuid: "9eb34beb910b4457a618e130cb7c8b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c73d576dc1304b8181c35475c71a35e4"
I20260927 18:39:28.448864 20095 raft_consensus.cc:3037] T 30d5b069f43844b8979287e078c63d21 P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:28.449716 20235 raft_consensus.cc:2454] T 30d5b069f43844b8979287e078c63d21 P 87bc6c3807a64aa2891412273b2eb0fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9eb34beb910b4457a618e130cb7c8b5e in term 1.
I20260927 18:39:28.449719 20095 raft_consensus.cc:2454] T 30d5b069f43844b8979287e078c63d21 P c73d576dc1304b8181c35475c71a35e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9eb34beb910b4457a618e130cb7c8b5e in term 1.
I20260927 18:39:28.449908 19888 leader_election.cc:302] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9eb34beb910b4457a618e130cb7c8b5e, c73d576dc1304b8181c35475c71a35e4; no voters:
I20260927 18:39:28.450037 20319 raft_consensus.cc:2781] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:28.450137 20319 raft_consensus.cc:686] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e [term 1 LEADER]: Becoming Leader. State: Replica: 9eb34beb910b4457a618e130cb7c8b5e, State: Running, Role: LEADER
I20260927 18:39:28.450202 20319 consensus_queue.cc:237] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.450564 20327 tablet_replica.cc:439] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e: TxnStatusTablet state changed. Reason: New leader 9eb34beb910b4457a618e130cb7c8b5e. Latest consensus state: current_term: 1 leader_uuid: "9eb34beb910b4457a618e130cb7c8b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } } }
I20260927 18:39:28.450640 20327 tablet_replica.cc:442] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:39:28.450804 20351 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 18:39:28.450940 20351 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 18:39:28.450941 19796 catalog_manager.cc:5495] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9eb34beb910b4457a618e130cb7c8b5e (127.11.251.193). New cstate: current_term: 1 leader_uuid: "9eb34beb910b4457a618e130cb7c8b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } health_report { overall_health: UNKNOWN } } }
I20260927 18:39:28.497085 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.494204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:42283 (local address 127.0.0.1:42522)
0927 18:39:28.494292 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:28.494307 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:28.494368 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:28.494546 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:28.494548 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:28.494549 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:39:28.494623 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:28.494628 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.495458 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.495459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:28.496084 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:28.496087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.496512 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.496514 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:28.496601 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.496618 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:39:28.496852 (+ 234us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:39:28.496863 (+ 11us) client_negotiation.cc:769] Sending connection context
0927 18:39:28.496925 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 18:39:28.497000 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260927 18:39:28.497072 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.494281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42522 (local address 127.11.251.193:42283)
0927 18:39:28.494341 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:28.494344 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:28.494356 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:39:28.494457 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:28.494459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:28.494474 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:28.494536 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:39:28.494714 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.495354 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.496197 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.496415 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.496508 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.496728 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:39:28.496779 (+ 51us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:39:28.496851 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:28.496931 (+ 80us) server_negotiation.cc:299] Negotiation successful
0927 18:39:28.496963 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 18:39:28.498446 20235 raft_consensus.cc:1261] T 30d5b069f43844b8979287e078c63d21 P 87bc6c3807a64aa2891412273b2eb0fd [term 1 FOLLOWER]: Refusing update from remote peer 9eb34beb910b4457a618e130cb7c8b5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:28.498689 20095 raft_consensus.cc:1261] T 30d5b069f43844b8979287e078c63d21 P c73d576dc1304b8181c35475c71a35e4 [term 1 FOLLOWER]: Refusing update from remote peer 9eb34beb910b4457a618e130cb7c8b5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:28.498724 20327 consensus_queue.cc:1035] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:28.498924 20327 consensus_queue.cc:1035] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:28.499471 20315 tablet_replica.cc:439] T 30d5b069f43844b8979287e078c63d21 P 87bc6c3807a64aa2891412273b2eb0fd: TxnStatusTablet state changed. Reason: New leader 9eb34beb910b4457a618e130cb7c8b5e. Latest consensus state: current_term: 1 leader_uuid: "9eb34beb910b4457a618e130cb7c8b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } } }
I20260927 18:39:28.499579 20315 tablet_replica.cc:442] T 30d5b069f43844b8979287e078c63d21 P 87bc6c3807a64aa2891412273b2eb0fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:39:28.500313 20319 tablet_replica.cc:439] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9eb34beb910b4457a618e130cb7c8b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } } }
I20260927 18:39:28.500417 20319 tablet_replica.cc:442] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:39:28.500715 20315 tablet_replica.cc:439] T 30d5b069f43844b8979287e078c63d21 P 87bc6c3807a64aa2891412273b2eb0fd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9eb34beb910b4457a618e130cb7c8b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } } }
I20260927 18:39:28.500828 20315 tablet_replica.cc:442] T 30d5b069f43844b8979287e078c63d21 P 87bc6c3807a64aa2891412273b2eb0fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:39:28.501453 20317 tablet_replica.cc:439] T 30d5b069f43844b8979287e078c63d21 P c73d576dc1304b8181c35475c71a35e4: TxnStatusTablet state changed. Reason: New leader 9eb34beb910b4457a618e130cb7c8b5e. Latest consensus state: current_term: 1 leader_uuid: "9eb34beb910b4457a618e130cb7c8b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } } }
I20260927 18:39:28.501566 20317 tablet_replica.cc:442] T 30d5b069f43844b8979287e078c63d21 P c73d576dc1304b8181c35475c71a35e4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:39:28.502233 20316 tablet_replica.cc:439] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9eb34beb910b4457a618e130cb7c8b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } } }
I20260927 18:39:28.502341 20316 tablet_replica.cc:442] T 30d5b069f43844b8979287e078c63d21 P 9eb34beb910b4457a618e130cb7c8b5e: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:39:28.502514 20317 tablet_replica.cc:439] T 30d5b069f43844b8979287e078c63d21 P c73d576dc1304b8181c35475c71a35e4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9eb34beb910b4457a618e130cb7c8b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } } }
I20260927 18:39:28.502604 20317 tablet_replica.cc:442] T 30d5b069f43844b8979287e078c63d21 P c73d576dc1304b8181c35475c71a35e4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:39:28.505424 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.502638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46016 (local address 127.11.251.195:33395)
0927 18:39:28.502686 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:28.502688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:28.502696 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:39:28.502732 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:28.502733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:28.502756 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:28.502825 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:39:28.503173 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.503717 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.504545 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.504780 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.504873 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.505041 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:39:28.505167 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:39:28.505262 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:28.505306 (+ 44us) server_negotiation.cc:299] Negotiation successful
0927 18:39:28.505331 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 18:39:28.510808 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.508217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:44049 (local address 127.11.251.193:42283)
0927 18:39:28.508268 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:28.508270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:28.508411 (+ 141us) server_negotiation.cc:407] Connection header received
0927 18:39:28.508465 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:28.508465 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:28.508475 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:28.508528 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:39:28.508647 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.509085 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.510092 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.510299 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.510361 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.510562 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:39:28.510592 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:39:28.510653 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:28.510737 (+ 84us) server_negotiation.cc:299] Negotiation successful
0927 18:39:28.510757 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 18:39:28.510859 20358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.508112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:42283 (local address 127.11.251.195:44049)
0927 18:39:28.508329 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:28.508340 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:28.508407 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:28.508530 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:28.508531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:28.508533 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:39:28.508589 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:28.508591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.509160 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.509161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:28.510009 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:28.510012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.510385 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.510386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:28.510450 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.510463 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:39:28.510677 (+ 214us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:39:28.510679 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:39:28.510734 (+ 55us) client_negotiation.cc:240] Negotiation successful
0927 18:39:28.510779 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":128,"threads_started":1}
I20260927 18:39:28.513633 19955 raft_consensus.cc:1261] T ae248dc03b584d4b8e45bc933a2d619b P 9eb34beb910b4457a618e130cb7c8b5e [term 1 FOLLOWER]: Refusing update from remote peer 87bc6c3807a64aa2891412273b2eb0fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:28.513622 20095 raft_consensus.cc:1261] T ae248dc03b584d4b8e45bc933a2d619b P c73d576dc1304b8181c35475c71a35e4 [term 1 FOLLOWER]: Refusing update from remote peer 87bc6c3807a64aa2891412273b2eb0fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:28.513912 20315 consensus_queue.cc:1035] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:28.514240 20339 consensus_queue.cc:1035] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:28.515499 20323 mvcc.cc:205] Tried to move back new op lower bound from 7334028773429661696 to 7334028772199841792. Current Snapshot: MvccSnapshot[applied={T|T < 7334028773429661696}]
I20260927 18:39:28.523314 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.520586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:36055 (local address 127.11.251.195:33395)
0927 18:39:28.520641 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:28.520644 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:28.520759 (+ 115us) server_negotiation.cc:407] Connection header received
0927 18:39:28.520861 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:28.520862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:28.520872 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:28.520909 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:39:28.521031 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.521497 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.522609 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.522837 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.522893 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.523090 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:39:28.523118 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:39:28.523158 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:28.523241 (+ 83us) server_negotiation.cc:299] Negotiation successful
0927 18:39:28.523259 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:39:28.523373 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.520528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:33395 (local address 127.11.251.193:36055)
0927 18:39:28.520686 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:28.520699 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:28.520788 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:28.520913 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:28.520913 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:28.520915 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:39:28.520970 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:28.520973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.521596 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.521597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:28.522496 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:28.522499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.522917 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.522919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:28.522977 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.522994 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:39:28.523182 (+ 188us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:39:28.523184 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:39:28.523239 (+ 55us) client_negotiation.cc:240] Negotiation successful
0927 18:39:28.523288 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":94,"threads_started":1}
I20260927 18:39:28.534315 19796 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:28.534448 19796 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:28.535223 19796 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad: Sending DeleteTablet for 3 replicas of tablet ae248dc03b584d4b8e45bc933a2d619b
I20260927 18:39:28.535492 20215 tablet_service.cc:1514] Processing DeleteTablet for tablet ae248dc03b584d4b8e45bc933a2d619b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:28 UTC) from {username='oryx', principal='oryx/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:46000
I20260927 18:39:28.535594 20333 tablet_replica.cc:330] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd: stopping tablet replica
I20260927 18:39:28.535666 20333 raft_consensus.cc:2229] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:39:28.535646 20075 tablet_service.cc:1514] Processing DeleteTablet for tablet ae248dc03b584d4b8e45bc933a2d619b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:28 UTC) from {username='oryx', principal='oryx/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:37318
I20260927 18:39:28.535756 20333 raft_consensus.cc:2258] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:28.535753 20334 tablet_replica.cc:330] T ae248dc03b584d4b8e45bc933a2d619b P c73d576dc1304b8181c35475c71a35e4: stopping tablet replica
I20260927 18:39:28.535753 19935 tablet_service.cc:1514] Processing DeleteTablet for tablet ae248dc03b584d4b8e45bc933a2d619b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:28 UTC) from {username='oryx', principal='oryx/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:42516
I20260927 18:39:28.535812 20334 raft_consensus.cc:2229] T ae248dc03b584d4b8e45bc933a2d619b P c73d576dc1304b8181c35475c71a35e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:28.535841 20332 tablet_replica.cc:330] T ae248dc03b584d4b8e45bc933a2d619b P 9eb34beb910b4457a618e130cb7c8b5e: stopping tablet replica
I20260927 18:39:28.535856 20334 raft_consensus.cc:2258] T ae248dc03b584d4b8e45bc933a2d619b P c73d576dc1304b8181c35475c71a35e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:28.535900 20332 raft_consensus.cc:2229] T ae248dc03b584d4b8e45bc933a2d619b P 9eb34beb910b4457a618e130cb7c8b5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:28.535938 20332 raft_consensus.cc:2258] T ae248dc03b584d4b8e45bc933a2d619b P 9eb34beb910b4457a618e130cb7c8b5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:28.536154 20333 ts_tablet_manager.cc:1905] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:28.536303 20334 ts_tablet_manager.cc:1905] T ae248dc03b584d4b8e45bc933a2d619b P c73d576dc1304b8181c35475c71a35e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:28.538182 20334 ts_tablet_manager.cc:1918] T ae248dc03b584d4b8e45bc933a2d619b P c73d576dc1304b8181c35475c71a35e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:39:28.538244 20334 log.cc:1196] T ae248dc03b584d4b8e45bc933a2d619b P c73d576dc1304b8181c35475c71a35e4: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/ae248dc03b584d4b8e45bc933a2d619b
I20260927 18:39:28.538352 20333 ts_tablet_manager.cc:1918] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:39:28.538405 20333 log.cc:1196] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/ae248dc03b584d4b8e45bc933a2d619b
I20260927 18:39:28.538467 20334 ts_tablet_manager.cc:1939] T ae248dc03b584d4b8e45bc933a2d619b P c73d576dc1304b8181c35475c71a35e4: Deleting consensus metadata
I20260927 18:39:28.538657 20333 ts_tablet_manager.cc:1939] T ae248dc03b584d4b8e45bc933a2d619b P 87bc6c3807a64aa2891412273b2eb0fd: Deleting consensus metadata
I20260927 18:39:28.538874 19784 catalog_manager.cc:4841] TS c73d576dc1304b8181c35475c71a35e4 (127.11.251.194:43985): tablet ae248dc03b584d4b8e45bc933a2d619b (table test-table [id=92b4d69b1d324fbaaa530a25f2fb6970]) successfully deleted
I20260927 18:39:28.539153 19783 catalog_manager.cc:4841] TS 87bc6c3807a64aa2891412273b2eb0fd (127.11.251.195:33395): tablet ae248dc03b584d4b8e45bc933a2d619b (table test-table [id=92b4d69b1d324fbaaa530a25f2fb6970]) successfully deleted
I20260927 18:39:28.540530 20332 ts_tablet_manager.cc:1905] T ae248dc03b584d4b8e45bc933a2d619b P 9eb34beb910b4457a618e130cb7c8b5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:28.542358 20332 ts_tablet_manager.cc:1918] T ae248dc03b584d4b8e45bc933a2d619b P 9eb34beb910b4457a618e130cb7c8b5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:39:28.542412 20332 log.cc:1196] T ae248dc03b584d4b8e45bc933a2d619b P 9eb34beb910b4457a618e130cb7c8b5e: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-0/wal/wals/ae248dc03b584d4b8e45bc933a2d619b
I20260927 18:39:28.542632 20332 ts_tablet_manager.cc:1939] T ae248dc03b584d4b8e45bc933a2d619b P 9eb34beb910b4457a618e130cb7c8b5e: Deleting consensus metadata
I20260927 18:39:28.543018 19785 catalog_manager.cc:4841] TS 9eb34beb910b4457a618e130cb7c8b5e (127.11.251.193:42283): tablet ae248dc03b584d4b8e45bc933a2d619b (table test-table [id=92b4d69b1d324fbaaa530a25f2fb6970]) successfully deleted
Sep 27 18:39:28 dist-test-slave-z47j krb5kdc[19753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 18:39:28 dist-test-slave-z47j krb5kdc[19753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:28 dist-test-slave-z47j krb5kdc[19753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
W20260927 18:39:28.571699 19848 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:28.564479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53718 (local address 127.11.251.254:46095)
0927 18:39:28.564534 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:28.564537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:28.564670 (+ 133us) server_negotiation.cc:407] Connection header received
0927 18:39:28.564739 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:28.564740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:28.564753 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:28.564830 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:28.565562 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.566445 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.567511 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.567870 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.567946 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.571616 (+ 3670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53718: BlockingRecv error: recv got EOF from 127.0.0.1:53718 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Sep 27 18:39:28 dist-test-slave-z47j krb5kdc[19753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM
I20260927 18:39:28.581947 19848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.572832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53728 (local address 127.11.251.254:46095)
0927 18:39:28.572879 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:28.572881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:28.573129 (+ 248us) server_negotiation.cc:407] Connection header received
0927 18:39:28.573182 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:28.573184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:28.573197 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:28.573264 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:28.573937 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.574731 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.575722 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.576041 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.576114 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.578841 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:28.578842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:28.578843 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:28.578850 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:28.580574 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:28.580968 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:28.580969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:28.580969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:28.581063 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:28.581281 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:28.581282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:28.581282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:28.581456 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:28.581600 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:28.581749 (+ 149us) server_negotiation.cc:299] Negotiation successful
0927 18:39:28.581834 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:39:28.583885 19796 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:39:28.584124 19796 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:39:28.586030 20215 tablet_service.cc:1467] Processing CreateTablet for tablet 7152355ca5af48adaabf993c37c090e8 (DEFAULT_TABLE table=test-table [id=69ae4b3b285b4ec0badf704646a1a470]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:28.586014 20075 tablet_service.cc:1467] Processing CreateTablet for tablet 7152355ca5af48adaabf993c37c090e8 (DEFAULT_TABLE table=test-table [id=69ae4b3b285b4ec0badf704646a1a470]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:28.586201 20075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7152355ca5af48adaabf993c37c090e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:28.586207 20215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7152355ca5af48adaabf993c37c090e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:28.587816 20309 tablet_bootstrap.cc:492] T 7152355ca5af48adaabf993c37c090e8 P 87bc6c3807a64aa2891412273b2eb0fd: Bootstrap starting.
I20260927 18:39:28.587896 19935 tablet_service.cc:1467] Processing CreateTablet for tablet 7152355ca5af48adaabf993c37c090e8 (DEFAULT_TABLE table=test-table [id=69ae4b3b285b4ec0badf704646a1a470]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:28.588037 19935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7152355ca5af48adaabf993c37c090e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:28.588302 20309 tablet_bootstrap.cc:654] T 7152355ca5af48adaabf993c37c090e8 P 87bc6c3807a64aa2891412273b2eb0fd: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:28.589305 20309 tablet_bootstrap.cc:492] T 7152355ca5af48adaabf993c37c090e8 P 87bc6c3807a64aa2891412273b2eb0fd: No bootstrap required, opened a new log
I20260927 18:39:28.589358 20309 ts_tablet_manager.cc:1397] T 7152355ca5af48adaabf993c37c090e8 P 87bc6c3807a64aa2891412273b2eb0fd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:39:28.589515 20311 tablet_bootstrap.cc:492] T 7152355ca5af48adaabf993c37c090e8 P c73d576dc1304b8181c35475c71a35e4: Bootstrap starting.
I20260927 18:39:28.589510 20309 raft_consensus.cc:348] T 7152355ca5af48adaabf993c37c090e8 P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.589637 20309 raft_consensus.cc:374] T 7152355ca5af48adaabf993c37c090e8 P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:28.589661 20309 raft_consensus.cc:729] T 7152355ca5af48adaabf993c37c090e8 P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87bc6c3807a64aa2891412273b2eb0fd, State: Initialized, Role: FOLLOWER
I20260927 18:39:28.589705 20309 consensus_queue.cc:260] T 7152355ca5af48adaabf993c37c090e8 P 87bc6c3807a64aa2891412273b2eb0fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.589797 20309 ts_tablet_manager.cc:1428] T 7152355ca5af48adaabf993c37c090e8 P 87bc6c3807a64aa2891412273b2eb0fd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:28.589998 20311 tablet_bootstrap.cc:654] T 7152355ca5af48adaabf993c37c090e8 P c73d576dc1304b8181c35475c71a35e4: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:28.590394 20310 tablet_bootstrap.cc:492] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e: Bootstrap starting.
I20260927 18:39:28.590822 20310 tablet_bootstrap.cc:654] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:28.591948 20310 tablet_bootstrap.cc:492] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e: No bootstrap required, opened a new log
I20260927 18:39:28.591997 20310 ts_tablet_manager.cc:1397] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 18:39:28.592181 20310 raft_consensus.cc:348] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.592242 20310 raft_consensus.cc:374] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:28.592259 20310 raft_consensus.cc:729] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eb34beb910b4457a618e130cb7c8b5e, State: Initialized, Role: FOLLOWER
I20260927 18:39:28.592298 20310 consensus_queue.cc:260] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.592382 20310 ts_tablet_manager.cc:1428] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:28.593287 20311 tablet_bootstrap.cc:492] T 7152355ca5af48adaabf993c37c090e8 P c73d576dc1304b8181c35475c71a35e4: No bootstrap required, opened a new log
I20260927 18:39:28.593336 20311 ts_tablet_manager.cc:1397] T 7152355ca5af48adaabf993c37c090e8 P c73d576dc1304b8181c35475c71a35e4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260927 18:39:28.593495 20311 raft_consensus.cc:348] T 7152355ca5af48adaabf993c37c090e8 P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.593562 20311 raft_consensus.cc:374] T 7152355ca5af48adaabf993c37c090e8 P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:28.593603 20311 raft_consensus.cc:729] T 7152355ca5af48adaabf993c37c090e8 P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c73d576dc1304b8181c35475c71a35e4, State: Initialized, Role: FOLLOWER
I20260927 18:39:28.593645 20311 consensus_queue.cc:260] T 7152355ca5af48adaabf993c37c090e8 P c73d576dc1304b8181c35475c71a35e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.593725 20311 ts_tablet_manager.cc:1428] T 7152355ca5af48adaabf993c37c090e8 P c73d576dc1304b8181c35475c71a35e4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:28.686125 20319 raft_consensus.cc:482] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:39:28.686260 20319 raft_consensus.cc:504] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.686604 20319 leader_election.cc:288] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c73d576dc1304b8181c35475c71a35e4 (127.11.251.194:43985), 87bc6c3807a64aa2891412273b2eb0fd (127.11.251.195:33395)
I20260927 18:39:28.686966 20095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7152355ca5af48adaabf993c37c090e8" candidate_uuid: "9eb34beb910b4457a618e130cb7c8b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c73d576dc1304b8181c35475c71a35e4" is_pre_election: true
I20260927 18:39:28.687101 20095 raft_consensus.cc:2454] T 7152355ca5af48adaabf993c37c090e8 P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9eb34beb910b4457a618e130cb7c8b5e in term 0.
I20260927 18:39:28.687067 20235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7152355ca5af48adaabf993c37c090e8" candidate_uuid: "9eb34beb910b4457a618e130cb7c8b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87bc6c3807a64aa2891412273b2eb0fd" is_pre_election: true
I20260927 18:39:28.687242 20235 raft_consensus.cc:2454] T 7152355ca5af48adaabf993c37c090e8 P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9eb34beb910b4457a618e130cb7c8b5e in term 0.
I20260927 18:39:28.687299 19888 leader_election.cc:302] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9eb34beb910b4457a618e130cb7c8b5e, c73d576dc1304b8181c35475c71a35e4; no voters:
I20260927 18:39:28.687407 20319 raft_consensus.cc:2781] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:39:28.687455 20319 raft_consensus.cc:482] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:39:28.687486 20319 raft_consensus.cc:3037] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:28.688391 20319 raft_consensus.cc:504] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.688544 20319 leader_election.cc:288] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e [CANDIDATE]: Term 1 election: Requested vote from peers c73d576dc1304b8181c35475c71a35e4 (127.11.251.194:43985), 87bc6c3807a64aa2891412273b2eb0fd (127.11.251.195:33395)
I20260927 18:39:28.688721 20095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7152355ca5af48adaabf993c37c090e8" candidate_uuid: "9eb34beb910b4457a618e130cb7c8b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c73d576dc1304b8181c35475c71a35e4"
I20260927 18:39:28.688810 20095 raft_consensus.cc:3037] T 7152355ca5af48adaabf993c37c090e8 P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:28.688805 20235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7152355ca5af48adaabf993c37c090e8" candidate_uuid: "9eb34beb910b4457a618e130cb7c8b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87bc6c3807a64aa2891412273b2eb0fd"
I20260927 18:39:28.688915 20235 raft_consensus.cc:3037] T 7152355ca5af48adaabf993c37c090e8 P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:28.689558 20095 raft_consensus.cc:2454] T 7152355ca5af48adaabf993c37c090e8 P c73d576dc1304b8181c35475c71a35e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9eb34beb910b4457a618e130cb7c8b5e in term 1.
I20260927 18:39:28.689725 20235 raft_consensus.cc:2454] T 7152355ca5af48adaabf993c37c090e8 P 87bc6c3807a64aa2891412273b2eb0fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9eb34beb910b4457a618e130cb7c8b5e in term 1.
I20260927 18:39:28.689795 19888 leader_election.cc:302] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9eb34beb910b4457a618e130cb7c8b5e, c73d576dc1304b8181c35475c71a35e4; no voters:
I20260927 18:39:28.689905 20319 raft_consensus.cc:2781] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:28.689970 20319 raft_consensus.cc:686] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e [term 1 LEADER]: Becoming Leader. State: Replica: 9eb34beb910b4457a618e130cb7c8b5e, State: Running, Role: LEADER
I20260927 18:39:28.690024 20319 consensus_queue.cc:237] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.690814 19796 catalog_manager.cc:5495] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9eb34beb910b4457a618e130cb7c8b5e (127.11.251.193). New cstate: current_term: 1 leader_uuid: "9eb34beb910b4457a618e130cb7c8b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } health_report { overall_health: UNKNOWN } } }
I20260927 18:39:28.697098 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.694665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42528 (local address 127.11.251.193:42283)
0927 18:39:28.694724 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:28.694726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:28.694758 (+ 32us) server_negotiation.cc:407] Connection header received
0927 18:39:28.694789 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:28.694790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:28.694800 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:28.694846 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:39:28.695013 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.695577 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.696383 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.696630 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.696691 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.696845 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:39:28.696889 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:39:28.696947 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:28.697003 (+ 56us) server_negotiation.cc:299] Negotiation successful
0927 18:39:28.697024 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 18:39:28.698035 20235 raft_consensus.cc:1261] T 7152355ca5af48adaabf993c37c090e8 P 87bc6c3807a64aa2891412273b2eb0fd [term 1 FOLLOWER]: Refusing update from remote peer 9eb34beb910b4457a618e130cb7c8b5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:28.698135 20095 raft_consensus.cc:1261] T 7152355ca5af48adaabf993c37c090e8 P c73d576dc1304b8181c35475c71a35e4 [term 1 FOLLOWER]: Refusing update from remote peer 9eb34beb910b4457a618e130cb7c8b5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:28.698292 20327 consensus_queue.cc:1035] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:28.698518 20327 consensus_queue.cc:1035] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:28.701682 19796 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:28.701786 19796 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:28.702620 19796 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad: Sending DeleteTablet for 3 replicas of tablet 7152355ca5af48adaabf993c37c090e8
I20260927 18:39:28.702888 19935 tablet_service.cc:1514] Processing DeleteTablet for tablet 7152355ca5af48adaabf993c37c090e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:28 UTC) from {username='oryx', principal='oryx/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:42516
I20260927 18:39:28.702924 20215 tablet_service.cc:1514] Processing DeleteTablet for tablet 7152355ca5af48adaabf993c37c090e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:28 UTC) from {username='oryx', principal='oryx/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:46000
I20260927 18:39:28.702996 20075 tablet_service.cc:1514] Processing DeleteTablet for tablet 7152355ca5af48adaabf993c37c090e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:28 UTC) from {username='oryx', principal='oryx/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:37318
I20260927 18:39:28.703050 20333 tablet_replica.cc:330] T 7152355ca5af48adaabf993c37c090e8 P 87bc6c3807a64aa2891412273b2eb0fd: stopping tablet replica
I20260927 18:39:28.703086 20334 tablet_replica.cc:330] T 7152355ca5af48adaabf993c37c090e8 P c73d576dc1304b8181c35475c71a35e4: stopping tablet replica
I20260927 18:39:28.703088 20332 tablet_replica.cc:330] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e: stopping tablet replica
I20260927 18:39:28.703107 20333 raft_consensus.cc:2229] T 7152355ca5af48adaabf993c37c090e8 P 87bc6c3807a64aa2891412273b2eb0fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:28.703145 20332 raft_consensus.cc:2229] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:39:28.703159 20333 raft_consensus.cc:2258] T 7152355ca5af48adaabf993c37c090e8 P 87bc6c3807a64aa2891412273b2eb0fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:28.703168 20334 raft_consensus.cc:2229] T 7152355ca5af48adaabf993c37c090e8 P c73d576dc1304b8181c35475c71a35e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:28.703202 20332 raft_consensus.cc:2258] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:28.703215 20334 raft_consensus.cc:2258] T 7152355ca5af48adaabf993c37c090e8 P c73d576dc1304b8181c35475c71a35e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:28.703130 19796 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:39:28.703301 19796 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:39:28.703516 20334 ts_tablet_manager.cc:1905] T 7152355ca5af48adaabf993c37c090e8 P c73d576dc1304b8181c35475c71a35e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:28.703524 20332 ts_tablet_manager.cc:1905] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:28.703728 20333 ts_tablet_manager.cc:1905] T 7152355ca5af48adaabf993c37c090e8 P 87bc6c3807a64aa2891412273b2eb0fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:28.705351 20332 ts_tablet_manager.cc:1918] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:39:28.705399 20334 ts_tablet_manager.cc:1918] T 7152355ca5af48adaabf993c37c090e8 P c73d576dc1304b8181c35475c71a35e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:39:28.705413 20332 log.cc:1196] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-0/wal/wals/7152355ca5af48adaabf993c37c090e8
I20260927 18:39:28.705457 20334 log.cc:1196] T 7152355ca5af48adaabf993c37c090e8 P c73d576dc1304b8181c35475c71a35e4: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/7152355ca5af48adaabf993c37c090e8
I20260927 18:39:28.705703 20332 ts_tablet_manager.cc:1939] T 7152355ca5af48adaabf993c37c090e8 P 9eb34beb910b4457a618e130cb7c8b5e: Deleting consensus metadata
I20260927 18:39:28.705760 20334 ts_tablet_manager.cc:1939] T 7152355ca5af48adaabf993c37c090e8 P c73d576dc1304b8181c35475c71a35e4: Deleting consensus metadata
I20260927 18:39:28.706029 19785 catalog_manager.cc:4841] TS 9eb34beb910b4457a618e130cb7c8b5e (127.11.251.193:42283): tablet 7152355ca5af48adaabf993c37c090e8 (table test-table [id=69ae4b3b285b4ec0badf704646a1a470]) successfully deleted
I20260927 18:39:28.706122 20333 ts_tablet_manager.cc:1918] T 7152355ca5af48adaabf993c37c090e8 P 87bc6c3807a64aa2891412273b2eb0fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:39:28.706187 20333 log.cc:1196] T 7152355ca5af48adaabf993c37c090e8 P 87bc6c3807a64aa2891412273b2eb0fd: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/7152355ca5af48adaabf993c37c090e8
I20260927 18:39:28.706190 19784 catalog_manager.cc:4841] TS c73d576dc1304b8181c35475c71a35e4 (127.11.251.194:43985): tablet 7152355ca5af48adaabf993c37c090e8 (table test-table [id=69ae4b3b285b4ec0badf704646a1a470]) successfully deleted
I20260927 18:39:28.706416 20333 ts_tablet_manager.cc:1939] T 7152355ca5af48adaabf993c37c090e8 P 87bc6c3807a64aa2891412273b2eb0fd: Deleting consensus metadata
I20260927 18:39:28.706739 19783 catalog_manager.cc:4841] TS 87bc6c3807a64aa2891412273b2eb0fd (127.11.251.195:33395): tablet 7152355ca5af48adaabf993c37c090e8 (table test-table [id=69ae4b3b285b4ec0badf704646a1a470]) successfully deleted
I20260927 18:39:28.707058 19935 tablet_service.cc:1467] Processing CreateTablet for tablet edb8a33b83434602983174c561a01231 (DEFAULT_TABLE table=test-table [id=9ff2ba68cfaa480da972c3e095b19e16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:28.707077 20215 tablet_service.cc:1467] Processing CreateTablet for tablet edb8a33b83434602983174c561a01231 (DEFAULT_TABLE table=test-table [id=9ff2ba68cfaa480da972c3e095b19e16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:28.707165 19935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edb8a33b83434602983174c561a01231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:28.707211 20215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edb8a33b83434602983174c561a01231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:28.708032 20310 tablet_bootstrap.cc:492] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e: Bootstrap starting.
I20260927 18:39:28.708071 20075 tablet_service.cc:1467] Processing CreateTablet for tablet edb8a33b83434602983174c561a01231 (DEFAULT_TABLE table=test-table [id=9ff2ba68cfaa480da972c3e095b19e16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:28.708128 20309 tablet_bootstrap.cc:492] T edb8a33b83434602983174c561a01231 P 87bc6c3807a64aa2891412273b2eb0fd: Bootstrap starting.
I20260927 18:39:28.708176 20075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edb8a33b83434602983174c561a01231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:28.708406 20310 tablet_bootstrap.cc:654] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:28.708443 20309 tablet_bootstrap.cc:654] T edb8a33b83434602983174c561a01231 P 87bc6c3807a64aa2891412273b2eb0fd: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:28.709309 20311 tablet_bootstrap.cc:492] T edb8a33b83434602983174c561a01231 P c73d576dc1304b8181c35475c71a35e4: Bootstrap starting.
I20260927 18:39:28.709848 20309 tablet_bootstrap.cc:492] T edb8a33b83434602983174c561a01231 P 87bc6c3807a64aa2891412273b2eb0fd: No bootstrap required, opened a new log
I20260927 18:39:28.709893 20309 ts_tablet_manager.cc:1397] T edb8a33b83434602983174c561a01231 P 87bc6c3807a64aa2891412273b2eb0fd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:39:28.710086 20311 tablet_bootstrap.cc:654] T edb8a33b83434602983174c561a01231 P c73d576dc1304b8181c35475c71a35e4: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:28.710100 20310 tablet_bootstrap.cc:492] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e: No bootstrap required, opened a new log
I20260927 18:39:28.710031 20309 raft_consensus.cc:348] T edb8a33b83434602983174c561a01231 P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.710131 20309 raft_consensus.cc:374] T edb8a33b83434602983174c561a01231 P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:28.710134 20310 ts_tablet_manager.cc:1397] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260927 18:39:28.710157 20309 raft_consensus.cc:729] T edb8a33b83434602983174c561a01231 P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87bc6c3807a64aa2891412273b2eb0fd, State: Initialized, Role: FOLLOWER
I20260927 18:39:28.710227 20309 consensus_queue.cc:260] T edb8a33b83434602983174c561a01231 P 87bc6c3807a64aa2891412273b2eb0fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.710276 20310 raft_consensus.cc:348] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.710348 20310 raft_consensus.cc:374] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:28.710369 20310 raft_consensus.cc:729] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eb34beb910b4457a618e130cb7c8b5e, State: Initialized, Role: FOLLOWER
I20260927 18:39:28.710361 20309 ts_tablet_manager.cc:1428] T edb8a33b83434602983174c561a01231 P 87bc6c3807a64aa2891412273b2eb0fd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:28.710414 20310 consensus_queue.cc:260] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.710515 20310 ts_tablet_manager.cc:1428] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:28.710841 20311 tablet_bootstrap.cc:492] T edb8a33b83434602983174c561a01231 P c73d576dc1304b8181c35475c71a35e4: No bootstrap required, opened a new log
I20260927 18:39:28.710891 20311 ts_tablet_manager.cc:1397] T edb8a33b83434602983174c561a01231 P c73d576dc1304b8181c35475c71a35e4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 18:39:28.711040 20311 raft_consensus.cc:348] T edb8a33b83434602983174c561a01231 P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.711117 20311 raft_consensus.cc:374] T edb8a33b83434602983174c561a01231 P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:28.711136 20311 raft_consensus.cc:729] T edb8a33b83434602983174c561a01231 P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c73d576dc1304b8181c35475c71a35e4, State: Initialized, Role: FOLLOWER
I20260927 18:39:28.711210 20311 consensus_queue.cc:260] T edb8a33b83434602983174c561a01231 P c73d576dc1304b8181c35475c71a35e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.711324 20311 ts_tablet_manager.cc:1428] T edb8a33b83434602983174c561a01231 P c73d576dc1304b8181c35475c71a35e4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:28.856562 20319 raft_consensus.cc:482] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:39:28.856719 20319 raft_consensus.cc:504] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.856956 20319 leader_election.cc:288] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c73d576dc1304b8181c35475c71a35e4 (127.11.251.194:43985), 87bc6c3807a64aa2891412273b2eb0fd (127.11.251.195:33395)
I20260927 18:39:28.857275 20235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edb8a33b83434602983174c561a01231" candidate_uuid: "9eb34beb910b4457a618e130cb7c8b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87bc6c3807a64aa2891412273b2eb0fd" is_pre_election: true
I20260927 18:39:28.857405 20235 raft_consensus.cc:2454] T edb8a33b83434602983174c561a01231 P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9eb34beb910b4457a618e130cb7c8b5e in term 0.
I20260927 18:39:28.857263 20095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edb8a33b83434602983174c561a01231" candidate_uuid: "9eb34beb910b4457a618e130cb7c8b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c73d576dc1304b8181c35475c71a35e4" is_pre_election: true
I20260927 18:39:28.857473 20095 raft_consensus.cc:2454] T edb8a33b83434602983174c561a01231 P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9eb34beb910b4457a618e130cb7c8b5e in term 0.
I20260927 18:39:28.857762 19887 leader_election.cc:302] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 87bc6c3807a64aa2891412273b2eb0fd, 9eb34beb910b4457a618e130cb7c8b5e, c73d576dc1304b8181c35475c71a35e4; no voters:
I20260927 18:39:28.857973 20319 raft_consensus.cc:2781] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:39:28.858043 20319 raft_consensus.cc:482] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:39:28.858080 20319 raft_consensus.cc:3037] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:28.859046 20319 raft_consensus.cc:504] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.859217 20319 leader_election.cc:288] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e [CANDIDATE]: Term 1 election: Requested vote from peers c73d576dc1304b8181c35475c71a35e4 (127.11.251.194:43985), 87bc6c3807a64aa2891412273b2eb0fd (127.11.251.195:33395)
I20260927 18:39:28.859416 20095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edb8a33b83434602983174c561a01231" candidate_uuid: "9eb34beb910b4457a618e130cb7c8b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c73d576dc1304b8181c35475c71a35e4"
I20260927 18:39:28.859555 20095 raft_consensus.cc:3037] T edb8a33b83434602983174c561a01231 P c73d576dc1304b8181c35475c71a35e4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:28.859547 20235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edb8a33b83434602983174c561a01231" candidate_uuid: "9eb34beb910b4457a618e130cb7c8b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87bc6c3807a64aa2891412273b2eb0fd"
I20260927 18:39:28.859634 20235 raft_consensus.cc:3037] T edb8a33b83434602983174c561a01231 P 87bc6c3807a64aa2891412273b2eb0fd [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:28.860554 20095 raft_consensus.cc:2454] T edb8a33b83434602983174c561a01231 P c73d576dc1304b8181c35475c71a35e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9eb34beb910b4457a618e130cb7c8b5e in term 1.
I20260927 18:39:28.860554 20235 raft_consensus.cc:2454] T edb8a33b83434602983174c561a01231 P 87bc6c3807a64aa2891412273b2eb0fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9eb34beb910b4457a618e130cb7c8b5e in term 1.
I20260927 18:39:28.860831 19887 leader_election.cc:302] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87bc6c3807a64aa2891412273b2eb0fd, 9eb34beb910b4457a618e130cb7c8b5e; no voters:
I20260927 18:39:28.860950 20319 raft_consensus.cc:2781] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:28.861070 20319 raft_consensus.cc:686] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e [term 1 LEADER]: Becoming Leader. State: Replica: 9eb34beb910b4457a618e130cb7c8b5e, State: Running, Role: LEADER
I20260927 18:39:28.861145 20319 consensus_queue.cc:237] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } }
I20260927 18:39:28.861992 19796 catalog_manager.cc:5495] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9eb34beb910b4457a618e130cb7c8b5e (127.11.251.193). New cstate: current_term: 1 leader_uuid: "9eb34beb910b4457a618e130cb7c8b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eb34beb910b4457a618e130cb7c8b5e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 } health_report { overall_health: UNKNOWN } } }
I20260927 18:39:28.900388 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.897681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:42283 (local address 127.11.251.193:50825)
0927 18:39:28.897765 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:28.897787 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:28.897844 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:28.898088 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:28.898089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:28.898090 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:39:28.898141 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:28.898142 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.898868 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.898869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:28.899478 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:28.899480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.899855 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.899856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:28.899911 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.899919 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:39:28.900185 (+ 266us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:39:28.900188 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:39:28.900243 (+ 55us) client_negotiation.cc:240] Negotiation successful
0927 18:39:28.900303 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260927 18:39:28.900475 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.897769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:50825 (local address 127.11.251.193:42283)
0927 18:39:28.897947 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:28.897950 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:28.897961 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:39:28.898003 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:28.898003 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:28.898015 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:28.898072 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:39:28.898260 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.898783 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.899572 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:28.899760 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:28.899823 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:28.900002 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:39:28.900032 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:39:28.900133 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:28.900346 (+ 213us) server_negotiation.cc:299] Negotiation successful
0927 18:39:28.900370 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260927 18:39:28.902786 20095 raft_consensus.cc:1261] T edb8a33b83434602983174c561a01231 P c73d576dc1304b8181c35475c71a35e4 [term 1 FOLLOWER]: Refusing update from remote peer 9eb34beb910b4457a618e130cb7c8b5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:28.902837 20235 raft_consensus.cc:1261] T edb8a33b83434602983174c561a01231 P 87bc6c3807a64aa2891412273b2eb0fd [term 1 FOLLOWER]: Refusing update from remote peer 9eb34beb910b4457a618e130cb7c8b5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:28.903043 20319 consensus_queue.cc:1035] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c73d576dc1304b8181c35475c71a35e4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 43985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:28.903151 20316 consensus_queue.cc:1035] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "87bc6c3807a64aa2891412273b2eb0fd" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 33395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:28.914381 19798 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:28.914484 19798 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:28.915307 19798 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 464cdece19dc49ca88c32d90fa0a40ad: Sending DeleteTablet for 3 replicas of tablet edb8a33b83434602983174c561a01231
I20260927 18:39:28.915618 20215 tablet_service.cc:1514] Processing DeleteTablet for tablet edb8a33b83434602983174c561a01231 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:28 UTC) from {username='oryx', principal='oryx/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:46000
I20260927 18:39:28.915629 20075 tablet_service.cc:1514] Processing DeleteTablet for tablet edb8a33b83434602983174c561a01231 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:28 UTC) from {username='oryx', principal='oryx/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:37318
I20260927 18:39:28.915742 20333 tablet_replica.cc:330] T edb8a33b83434602983174c561a01231 P 87bc6c3807a64aa2891412273b2eb0fd: stopping tablet replica
I20260927 18:39:28.915755 20334 tablet_replica.cc:330] T edb8a33b83434602983174c561a01231 P c73d576dc1304b8181c35475c71a35e4: stopping tablet replica
I20260927 18:39:28.915802 20333 raft_consensus.cc:2229] T edb8a33b83434602983174c561a01231 P 87bc6c3807a64aa2891412273b2eb0fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:28.915822 20334 raft_consensus.cc:2229] T edb8a33b83434602983174c561a01231 P c73d576dc1304b8181c35475c71a35e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:28.915838 20333 raft_consensus.cc:2258] T edb8a33b83434602983174c561a01231 P 87bc6c3807a64aa2891412273b2eb0fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:28.915858 20334 raft_consensus.cc:2258] T edb8a33b83434602983174c561a01231 P c73d576dc1304b8181c35475c71a35e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:28.916090 19935 tablet_service.cc:1514] Processing DeleteTablet for tablet edb8a33b83434602983174c561a01231 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:28 UTC) from {username='oryx', principal='oryx/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:42516
I20260927 18:39:28.916143 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 19870
I20260927 18:39:28.916198 20332 tablet_replica.cc:330] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e: stopping tablet replica
I20260927 18:39:28.916278 20332 raft_consensus.cc:2229] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:39:28.916301 20333 ts_tablet_manager.cc:1905] T edb8a33b83434602983174c561a01231 P 87bc6c3807a64aa2891412273b2eb0fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:28.916358 20332 raft_consensus.cc:2258] T edb8a33b83434602983174c561a01231 P 9eb34beb910b4457a618e130cb7c8b5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:28.917222 20334 ts_tablet_manager.cc:1905] T edb8a33b83434602983174c561a01231 P c73d576dc1304b8181c35475c71a35e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:28.918226 20333 ts_tablet_manager.cc:1918] T edb8a33b83434602983174c561a01231 P 87bc6c3807a64aa2891412273b2eb0fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:39:28.918315 20333 log.cc:1196] T edb8a33b83434602983174c561a01231 P 87bc6c3807a64aa2891412273b2eb0fd: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/edb8a33b83434602983174c561a01231
I20260927 18:39:28.918551 20333 ts_tablet_manager.cc:1939] T edb8a33b83434602983174c561a01231 P 87bc6c3807a64aa2891412273b2eb0fd: Deleting consensus metadata
I20260927 18:39:28.918921 19783 catalog_manager.cc:4841] TS 87bc6c3807a64aa2891412273b2eb0fd (127.11.251.195:33395): tablet edb8a33b83434602983174c561a01231 (table test-table [id=9ff2ba68cfaa480da972c3e095b19e16]) successfully deleted
I20260927 18:39:28.919478 20334 ts_tablet_manager.cc:1918] T edb8a33b83434602983174c561a01231 P c73d576dc1304b8181c35475c71a35e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:39:28.919552 20334 log.cc:1196] T edb8a33b83434602983174c561a01231 P c73d576dc1304b8181c35475c71a35e4: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/edb8a33b83434602983174c561a01231
I20260927 18:39:28.919770 20334 ts_tablet_manager.cc:1939] T edb8a33b83434602983174c561a01231 P c73d576dc1304b8181c35475c71a35e4: Deleting consensus metadata
I20260927 18:39:28.921981 19784 catalog_manager.cc:4841] TS c73d576dc1304b8181c35475c71a35e4 (127.11.251.194:43985): tablet edb8a33b83434602983174c561a01231 (table test-table [id=9ff2ba68cfaa480da972c3e095b19e16]) successfully deleted
W20260927 18:39:28.925777 19785 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.251.193:42283 (error 108)
I20260927 18:39:28.926117 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 20010
I20260927 18:39:28.927882 20394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:28.927182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:42283 (local address 127.0.0.1:42542)
0927 18:39:28.927797 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:28.927839 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.193:42283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":46,"threads_started":1}
W20260927 18:39:28.928021 19785 catalog_manager.cc:4584] TS 9eb34beb910b4457a618e130cb7c8b5e (127.11.251.193:42283): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet edb8a33b83434602983174c561a01231: Network error: Client connection negotiation failed: client connection to 127.11.251.193:42283: connect: Connection refused (error 111)
I20260927 18:39:28.932641 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 20150
I20260927 18:39:28.940155 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 19766
2026-09-27T18:39:28Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2919 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:39:28 dist-test-slave-z47j krb5kdc[20397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:39:28 dist-test-slave-z47j krb5kdc[20397](info): set up 2 sockets
Sep 27 18:39:28 dist-test-slave-z47j krb5kdc[20397](info): commencing operation
krb5kdc: starting...
W20260927 18:39:29.994763 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:39:30 dist-test-slave-z47j krb5kdc[20397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:39:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:39:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:39:30.124770 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:41803
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:45933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:41803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:30.213625 20410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:30.213853 20410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:30.213881 20410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:30.215073 20410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:30.215112 20410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:30.215126 20410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:30.215139 20410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:30.215152 20410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:39:30.216570 20410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:45933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:41803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:41803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:30.216779 20410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:30.216981 20410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:30.222060 20418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:30.222107 20415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:30.222083 20416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:30.222370 20410 server_base.cc:1034] running on GCE node
I20260927 18:39:30.222674 20410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:30.223058 20410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:30.224216 20410 hybrid_clock.cc:648] HybridClock initialized: now 1790534370224202 us; error 37 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-27T18:39:30Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1289 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:39:30 dist-test-slave-z47j krb5kdc[20423](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:39:30 dist-test-slave-z47j krb5kdc[20423](info): set up 2 sockets
Sep 27 18:39:30 dist-test-slave-z47j krb5kdc[20423](info): commencing operation
krb5kdc: starting...
W20260927 18:39:31.274497 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:39:31 dist-test-slave-z47j krb5kdc[20423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:39:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:39:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:39:31.406093 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:41019
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:41019
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:31.494906 20436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:31.495114 20436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:31.495136 20436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:31.496273 20436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:31.496318 20436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:31.496332 20436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:31.496374 20436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:31.496410 20436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:39:31.497851 20436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:41019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:41019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:31.498099 20436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:31.498351 20436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:31.503319 20442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:31.503305 20441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:31.503312 20436 server_base.cc:1034] running on GCE node
W20260927 18:39:31.503361 20444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:31.503839 20436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:31.504199 20436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:31.505348 20436 hybrid_clock.cc:648] HybridClock initialized: now 1790534371505326 us; error 50 us; skew 500 ppm
Sep 27 18:39:31 dist-test-slave-z47j krb5kdc[20423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534371, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:31.508265 20436 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:39:31.508749 20436 webserver.cc:460] Webserver started at http://127.11.251.254:42851/ using document root <none> and password file <none>
I20260927 18:39:31.508980 20436 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:31.509032 20436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:31.509120 20436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:31.510164 20436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "3e5894f2d08a4894b54e12a5377ac9f9"
format_stamp: "Formatted at 2026-09-27 18:39:31 on dist-test-slave-z47j"
server_key: "5d6bbff6b330cd23a0ccb67873232954"
server_key_iv: "f8bb3384e8e761364e38bb54097e395b"
server_key_version: "encryptionkey@0"
I20260927 18:39:31.510478 20436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "3e5894f2d08a4894b54e12a5377ac9f9"
format_stamp: "Formatted at 2026-09-27 18:39:31 on dist-test-slave-z47j"
server_key: "5d6bbff6b330cd23a0ccb67873232954"
server_key_iv: "f8bb3384e8e761364e38bb54097e395b"
server_key_version: "encryptionkey@0"
I20260927 18:39:31.511749 20436 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:31.512472 20451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:31.512696 20436 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:31.512789 20436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "3e5894f2d08a4894b54e12a5377ac9f9"
format_stamp: "Formatted at 2026-09-27 18:39:31 on dist-test-slave-z47j"
server_key: "5d6bbff6b330cd23a0ccb67873232954"
server_key_iv: "f8bb3384e8e761364e38bb54097e395b"
server_key_version: "encryptionkey@0"
I20260927 18:39:31.512866 20436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:31.527079 20436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:31.529706 20436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:31.529888 20436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:31.534844 20436 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:41019
I20260927 18:39:31.534907 20503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:41019 every 8 connection(s)
I20260927 18:39:31.535319 20436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:39:31.536377 20504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:31.538936 20504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9: Bootstrap starting.
I20260927 18:39:31.539563 20504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:31.539875 20504 log.cc:824] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:31.540709 20504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9: No bootstrap required, opened a new log
I20260927 18:39:31.541800 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 20436
I20260927 18:39:31.541939 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:39:31.542161 12271 external_mini_cluster.cc:1428] Setting key 774195dc991ae7098ae69c525909037e
I20260927 18:39:31.543080 20504 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5894f2d08a4894b54e12a5377ac9f9" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41019 } }
I20260927 18:39:31.543228 20504 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:31.543253 20504 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5894f2d08a4894b54e12a5377ac9f9, State: Initialized, Role: FOLLOWER
I20260927 18:39:31.543397 20504 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5894f2d08a4894b54e12a5377ac9f9" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41019 } }
I20260927 18:39:31.543490 20504 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:39:31.543560 20504 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:39:31.543617 20504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:31.544539 20504 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5894f2d08a4894b54e12a5377ac9f9" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41019 } }
I20260927 18:39:31.544663 20504 leader_election.cc:302] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e5894f2d08a4894b54e12a5377ac9f9; no voters:
I20260927 18:39:31.544884 20504 leader_election.cc:288] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:39:31.544914 20509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:31.545027 20509 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [term 1 LEADER]: Becoming Leader. State: Replica: 3e5894f2d08a4894b54e12a5377ac9f9, State: Running, Role: LEADER
I20260927 18:39:31.545149 20504 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:31.545123 20509 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5894f2d08a4894b54e12a5377ac9f9" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41019 } }
I20260927 18:39:31.545495 20509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e5894f2d08a4894b54e12a5377ac9f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5894f2d08a4894b54e12a5377ac9f9" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41019 } } }
I20260927 18:39:31.545603 20509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:31.545759 20509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e5894f2d08a4894b54e12a5377ac9f9. Latest consensus state: current_term: 1 leader_uuid: "3e5894f2d08a4894b54e12a5377ac9f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5894f2d08a4894b54e12a5377ac9f9" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41019 } } }
I20260927 18:39:31.545849 20509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:31.546039 20515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:39:31.546808 20515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:39:31.549044 20515 catalog_manager.cc:1347] Generated new cluster ID: d0d18e9069274644bcfb942ee042fda7
I20260927 18:39:31.549113 20515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:39:31 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:31.553893 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:31.542773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32844 (local address 127.11.251.254:41019)
0927 18:39:31.542977 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:31.542986 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:31.543016 (+ 30us) server_negotiation.cc:407] Connection header received
0927 18:39:31.543137 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:31.543144 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:31.543198 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:31.543394 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:31.544192 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:31.545405 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:31.546847 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:31.547501 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:31.547641 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:31.550441 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:31.550445 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:31.550446 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:31.550458 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:31.552252 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:31.552626 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:31.552629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:31.552630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:31.552742 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:31.552966 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:31.552967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:31.552967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:31.553231 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:31.553407 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:31.553505 (+ 98us) server_negotiation.cc:299] Negotiation successful
0927 18:39:31.553683 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":112,"threads_started":1}
I20260927 18:39:31.570596 20515 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:39:31.571424 20515 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:39:31.576521 20515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9: Generated new TSK 0
I20260927 18:39:31.576740 20515 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.251.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
I20260927 18:39:31.613902 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41019
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:31.697383 20530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:31.697602 20530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:31.697626 20530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:31.698781 20530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:31.698823 20530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:31.698850 20530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:39:31.700198 20530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:31.700390 20530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:31.700575 20530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:31.705732 20536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:31.705749 20535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:31.705749 20538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:31.705868 20530 server_base.cc:1034] running on GCE node
I20260927 18:39:31.706250 20530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:31.706588 20530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:31.707834 20530 hybrid_clock.cc:648] HybridClock initialized: now 1790534371707817 us; error 28 us; skew 500 ppm
Sep 27 18:39:31 dist-test-slave-z47j krb5kdc[20423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534371, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:31.711094 20530 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:39:31.711612 20530 webserver.cc:460] Webserver started at http://127.11.251.193:38363/ using document root <none> and password file <none>
I20260927 18:39:31.711870 20530 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:31.711941 20530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:31.712097 20530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:31.713166 20530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "70e9a9dd5e714667ac05c35e315e9e13"
format_stamp: "Formatted at 2026-09-27 18:39:31 on dist-test-slave-z47j"
server_key: "56d7d0820a89db5e09bbf5c776ea0dbd"
server_key_iv: "7bdefec336da42d54b3d74097be024ae"
server_key_version: "encryptionkey@0"
I20260927 18:39:31.713635 20530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "70e9a9dd5e714667ac05c35e315e9e13"
format_stamp: "Formatted at 2026-09-27 18:39:31 on dist-test-slave-z47j"
server_key: "56d7d0820a89db5e09bbf5c776ea0dbd"
server_key_iv: "7bdefec336da42d54b3d74097be024ae"
server_key_version: "encryptionkey@0"
I20260927 18:39:31.715440 20530 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260927 18:39:31.716302 20545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:31.716540 20530 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:31.716630 20530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "70e9a9dd5e714667ac05c35e315e9e13"
format_stamp: "Formatted at 2026-09-27 18:39:31 on dist-test-slave-z47j"
server_key: "56d7d0820a89db5e09bbf5c776ea0dbd"
server_key_iv: "7bdefec336da42d54b3d74097be024ae"
server_key_version: "encryptionkey@0"
I20260927 18:39:31.716734 20530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:31.730260 20530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:31.732507 20530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:31.732681 20530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:31.732985 20530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:31.733349 20530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:31.733409 20530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:31.733444 20530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:31.733476 20530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:31.740448 20530 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:40297
I20260927 18:39:31.740500 20658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:40297 every 8 connection(s)
I20260927 18:39:31.740825 20530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
Sep 27 18:39:31 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534371, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:31.749341 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 20530
I20260927 18:39:31.749477 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:39:31.749718 12271 external_mini_cluster.cc:1428] Setting key 7cfdfaa820a3f1742391dfed5cc02797
I20260927 18:39:31.751240 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:31.741688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:60061 (local address 127.11.251.254:41019)
0927 18:39:31.741747 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:31.741749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:31.742015 (+ 266us) server_negotiation.cc:407] Connection header received
0927 18:39:31.742528 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:31.742530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:31.742549 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:31.742614 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:31.743536 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:31.744109 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:31.745362 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:31.745613 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:31.745713 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:31.748247 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:31.748248 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:31.748249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:31.748256 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:31.749832 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:31.750217 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:31.750219 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:31.750219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:31.750297 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:31.750518 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:31.750519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:31.750520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:31.750654 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:31.750714 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:31.751086 (+ 372us) server_negotiation.cc:299] Negotiation successful
0927 18:39:31.751165 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 18:39:31.751503 20661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:31.741680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.193:60061)
0927 18:39:31.741885 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:31.741923 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:31.742424 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:31.742670 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:31.742673 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:31.743242 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:31.743443 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:31.743447 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:31.744249 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:31.744250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:31.745277 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:31.745280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:31.745732 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:31.745733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:31.745916 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:31.746544 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:31.746553 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:31.748147 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:31.749922 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:31.749929 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:31.749933 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:31.750145 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:31.750366 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:31.750367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:31.750367 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:31.750464 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:31.750730 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:31.750737 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:31.750920 (+ 183us) client_negotiation.cc:769] Sending connection context
0927 18:39:31.751049 (+ 129us) client_negotiation.cc:240] Negotiation successful
0927 18:39:31.751229 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":98,"threads_started":1}
I20260927 18:39:31.752048 20659 heartbeater.cc:347] Connected to a master server at 127.11.251.254:41019
I20260927 18:39:31.752208 20659 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:31.752453 20659 heartbeater.cc:511] Master 127.11.251.254:41019 requested a full tablet report, sending...
I20260927 18:39:31.753094 20468 ts_manager.cc:194] Registered new tserver with Master: 70e9a9dd5e714667ac05c35e315e9e13 (127.11.251.193:40297)
I20260927 18:39:31.753870 20468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:60061
WARNING: no policy specified for kudu/127.11.251.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
I20260927 18:39:31.777158 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41019
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:31.867187 20664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:31.867465 20664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:31.867507 20664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:31.868724 20664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:31.868793 20664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:31.868836 20664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:39:31.870407 20664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:31.870688 20664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:31.870940 20664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:31.876436 20672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:31.876425 20669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:31.876508 20670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:31.876924 20664 server_base.cc:1034] running on GCE node
I20260927 18:39:31.877192 20664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:31.877466 20664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:31.878641 20664 hybrid_clock.cc:648] HybridClock initialized: now 1790534371878619 us; error 32 us; skew 500 ppm
Sep 27 18:39:31 dist-test-slave-z47j krb5kdc[20423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534371, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:31.881841 20664 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:39:31.882423 20664 webserver.cc:460] Webserver started at http://127.11.251.194:43819/ using document root <none> and password file <none>
I20260927 18:39:31.882715 20664 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:31.882789 20664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:31.882947 20664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:31.884150 20664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "f0b554de5ce74c8a8f8b27298456cfe8"
format_stamp: "Formatted at 2026-09-27 18:39:31 on dist-test-slave-z47j"
server_key: "2a100dab274c2e3f8232cbcada71a4d9"
server_key_iv: "3b59c3bf67a72c95274fe64ac0fd15dd"
server_key_version: "encryptionkey@0"
I20260927 18:39:31.884557 20664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "f0b554de5ce74c8a8f8b27298456cfe8"
format_stamp: "Formatted at 2026-09-27 18:39:31 on dist-test-slave-z47j"
server_key: "2a100dab274c2e3f8232cbcada71a4d9"
server_key_iv: "3b59c3bf67a72c95274fe64ac0fd15dd"
server_key_version: "encryptionkey@0"
I20260927 18:39:31.886142 20664 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 18:39:31.887042 20679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:31.887306 20664 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:31.887408 20664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "f0b554de5ce74c8a8f8b27298456cfe8"
format_stamp: "Formatted at 2026-09-27 18:39:31 on dist-test-slave-z47j"
server_key: "2a100dab274c2e3f8232cbcada71a4d9"
server_key_iv: "3b59c3bf67a72c95274fe64ac0fd15dd"
server_key_version: "encryptionkey@0"
I20260927 18:39:31.887497 20664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:31.903641 20664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:31.905995 20664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:31.906178 20664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:31.906455 20664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:31.906803 20664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:31.906858 20664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:31.906906 20664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:31.906937 20664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:31.913961 20664 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:42987
I20260927 18:39:31.914012 20792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:42987 every 8 connection(s)
I20260927 18:39:31.914383 20664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
Sep 27 18:39:31 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534371, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:31.923280 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 20664
I20260927 18:39:31.923429 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:39:31.923705 12271 external_mini_cluster.cc:1428] Setting key 003a27810d660415a818e1e0f05b8ef3
I20260927 18:39:31.924356 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:31.915199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:53929 (local address 127.11.251.254:41019)
0927 18:39:31.915325 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:31.915327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:31.915500 (+ 173us) server_negotiation.cc:407] Connection header received
0927 18:39:31.916061 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:31.916063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:31.916076 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:31.916170 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:31.917261 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:31.917875 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:31.918900 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:31.919133 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:31.919209 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:31.921634 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:31.921636 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:31.921636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:31.921643 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:31.922845 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:31.923220 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:31.923222 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:31.923222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:31.923290 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:31.923589 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:31.923590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:31.923590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:31.923739 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:31.923814 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:31.924168 (+ 354us) server_negotiation.cc:299] Negotiation successful
0927 18:39:31.924260 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260927 18:39:31.924580 20795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:31.915115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.194:53929)
0927 18:39:31.915330 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:31.915377 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:31.915961 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:31.916250 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:31.916254 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:31.916948 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:31.917177 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:31.917182 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:31.917957 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:31.917958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:31.918804 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:31.918805 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:31.919218 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:31.919218 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:31.919319 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:31.919928 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:31.919937 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:31.921494 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:31.922914 (+ 1420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:31.922920 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:31.922922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:31.923130 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:31.923381 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:31.923382 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:31.923382 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:31.923535 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:31.923847 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:31.923855 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:31.924007 (+ 152us) client_negotiation.cc:769] Sending connection context
0927 18:39:31.924132 (+ 125us) client_negotiation.cc:240] Negotiation successful
0927 18:39:31.924304 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":97,"threads_started":1}
I20260927 18:39:31.925058 20793 heartbeater.cc:347] Connected to a master server at 127.11.251.254:41019
I20260927 18:39:31.925186 20793 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:31.925382 20793 heartbeater.cc:511] Master 127.11.251.254:41019 requested a full tablet report, sending...
I20260927 18:39:31.925843 20468 ts_manager.cc:194] Registered new tserver with Master: f0b554de5ce74c8a8f8b27298456cfe8 (127.11.251.194:42987)
I20260927 18:39:31.926388 20468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:53929
WARNING: no policy specified for kudu/127.11.251.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
I20260927 18:39:31.953011 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41019
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:32.041379 20798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:32.041661 20798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:32.041697 20798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:32.042881 20798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:32.042912 20798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:32.042936 20798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:39:32.044312 20798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:32.044512 20798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:32.044787 20798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:32.050652 20806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:32.050676 20804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:32.050809 20798 server_base.cc:1034] running on GCE node
W20260927 18:39:32.050699 20803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:32.051163 20798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:32.051422 20798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:32.052577 20798 hybrid_clock.cc:648] HybridClock initialized: now 1790534372052556 us; error 39 us; skew 500 ppm
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534372, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:32.055392 20798 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:39:32.055905 20798 webserver.cc:460] Webserver started at http://127.11.251.195:37447/ using document root <none> and password file <none>
I20260927 18:39:32.056180 20798 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:32.056250 20798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:32.056408 20798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:32.057557 20798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "dacd9db039614d0bb455e4ca9d9151c9"
format_stamp: "Formatted at 2026-09-27 18:39:32 on dist-test-slave-z47j"
server_key: "0a5bdae0e32a6e33b6ad84b759c2acc9"
server_key_iv: "548995e0523fc27128c25b3c29f0b74c"
server_key_version: "encryptionkey@0"
I20260927 18:39:32.058007 20798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "dacd9db039614d0bb455e4ca9d9151c9"
format_stamp: "Formatted at 2026-09-27 18:39:32 on dist-test-slave-z47j"
server_key: "0a5bdae0e32a6e33b6ad84b759c2acc9"
server_key_iv: "548995e0523fc27128c25b3c29f0b74c"
server_key_version: "encryptionkey@0"
I20260927 18:39:32.059435 20798 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:39:32.060176 20813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:32.060392 20798 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:32.060497 20798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "dacd9db039614d0bb455e4ca9d9151c9"
format_stamp: "Formatted at 2026-09-27 18:39:32 on dist-test-slave-z47j"
server_key: "0a5bdae0e32a6e33b6ad84b759c2acc9"
server_key_iv: "548995e0523fc27128c25b3c29f0b74c"
server_key_version: "encryptionkey@0"
I20260927 18:39:32.060570 20798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:32.071838 20798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:32.073922 20798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:32.074086 20798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:32.074326 20798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:32.074651 20798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:32.074705 20798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:32.074750 20798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:32.074774 20798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:32.081525 20798 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:37135
I20260927 18:39:32.081596 20926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:37135 every 8 connection(s)
I20260927 18:39:32.081926 20798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534372, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:32.088937 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 20798
I20260927 18:39:32.089044 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:39:32.089241 12271 external_mini_cluster.cc:1428] Setting key 2071f0cac90044199c87ae9d73e886e3
I20260927 18:39:32.091882 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:32.082738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:34829 (local address 127.11.251.254:41019)
0927 18:39:32.082801 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:32.082803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:32.082891 (+ 88us) server_negotiation.cc:407] Connection header received
0927 18:39:32.083454 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:32.083456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:32.083465 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:32.083516 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:32.084509 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.085047 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.086192 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.086376 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.086432 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:32.088774 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:32.088777 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:32.088777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:32.088783 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:32.090525 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:32.090892 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:32.090893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:32.090894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:32.090949 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:32.091164 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:32.091164 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:32.091165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:32.091319 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:32.091382 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:32.091681 (+ 299us) server_negotiation.cc:299] Negotiation successful
0927 18:39:32.091777 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260927 18:39:32.092098 20929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:32.082573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.195:34829)
0927 18:39:32.082732 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:32.082784 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:32.083360 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:32.083584 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:32.083588 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:32.084204 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:32.084431 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:32.084436 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.085132 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.085133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:32.086092 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:32.086095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.086452 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.086453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:32.086566 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:32.087145 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:32.087170 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:32.088626 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:32.090591 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:32.090597 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:32.090599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:32.090822 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:32.091012 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:32.091012 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:32.091013 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:32.091110 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:32.091384 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:32.091388 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:32.091510 (+ 122us) client_negotiation.cc:769] Sending connection context
0927 18:39:32.091639 (+ 129us) client_negotiation.cc:240] Negotiation successful
0927 18:39:32.091843 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260927 18:39:32.092549 20927 heartbeater.cc:347] Connected to a master server at 127.11.251.254:41019
I20260927 18:39:32.092664 20927 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:32.092852 20927 heartbeater.cc:511] Master 127.11.251.254:41019 requested a full tablet report, sending...
I20260927 18:39:32.093236 20468 ts_manager.cc:194] Registered new tserver with Master: dacd9db039614d0bb455e4ca9d9151c9 (127.11.251.195:37135)
I20260927 18:39:32.093708 20468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:34829
I20260927 18:39:32.101433 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.251.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.251.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.251.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.193.keytab.
Entry for principal oryx/127.11.251.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.193.keytab.
WARNING: no policy specified for oryx/127.11.251.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.251.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.251.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.194.keytab.
Entry for principal oryx/127.11.251.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.194.keytab.
WARNING: no policy specified for oryx/127.11.251.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.251.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.251.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.195.keytab.
Entry for principal oryx/127.11.251.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.195.keytab.
I20260927 18:39:32.182158 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 20530
I20260927 18:39:32.188423 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 20664
I20260927 18:39:32.193233 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 20798
I20260927 18:39:32.197906 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 20436
I20260927 18:39:32.203888 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:41019
--webserver_interface=127.11.251.254
--webserver_port=42851
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:41019
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:32.293035 20936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:32.293286 20936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:32.293311 20936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:32.294601 20936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:32.294654 20936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:32.294669 20936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:32.294682 20936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:32.294694 20936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:39:32.296226 20936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:41019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:41019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=42851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:32.296525 20936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:32.296792 20936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:39:32.302186 20936 server_base.cc:1034] running on GCE node
W20260927 18:39:32.302156 20941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:32.302155 20944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:32.302141 20942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:32.302606 20936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:32.302942 20936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:32.304145 20936 hybrid_clock.cc:648] HybridClock initialized: now 1790534372304124 us; error 37 us; skew 500 ppm
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534372, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:32.306924 20936 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:39:32.307430 20936 webserver.cc:460] Webserver started at http://127.11.251.254:42851/ using document root <none> and password file <none>
I20260927 18:39:32.307670 20936 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:32.307750 20936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:32.309203 20936 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:39:32.309954 20951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:32.310177 20936 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 18:39:32.310272 20936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "3e5894f2d08a4894b54e12a5377ac9f9"
format_stamp: "Formatted at 2026-09-27 18:39:31 on dist-test-slave-z47j"
server_key: "5d6bbff6b330cd23a0ccb67873232954"
server_key_iv: "f8bb3384e8e761364e38bb54097e395b"
server_key_version: "encryptionkey@0"
I20260927 18:39:32.310688 20936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:32.322981 20936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:32.324805 20936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:32.324966 20936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:32.329463 20936 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:41019
I20260927 18:39:32.329536 21003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:41019 every 8 connection(s)
I20260927 18:39:32.329917 20936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:39:32.331523 21004 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:39:32.332121 21004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9: Bootstrap starting.
I20260927 18:39:32.339715 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 20936
I20260927 18:39:32.340183 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:40297
--local_ip_for_outbound_sockets=127.11.251.193
--tserver_master_addrs=127.11.251.254:41019
--webserver_port=38363
--webserver_interface=127.11.251.193
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.251.193
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:39:32.342895 21004 log.cc:824] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:32.345016 21004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:39:32.345364 21004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9: Bootstrap complete.
I20260927 18:39:32.347977 21004 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5894f2d08a4894b54e12a5377ac9f9" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41019 } }
I20260927 18:39:32.348248 21004 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5894f2d08a4894b54e12a5377ac9f9, State: Initialized, Role: FOLLOWER
I20260927 18:39:32.348441 21004 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5894f2d08a4894b54e12a5377ac9f9" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41019 } }
I20260927 18:39:32.348522 21004 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:39:32.348588 21004 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:39:32.348639 21004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260927 18:39:32.349908 21004 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5894f2d08a4894b54e12a5377ac9f9" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41019 } }
I20260927 18:39:32.349998 21004 leader_election.cc:302] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e5894f2d08a4894b54e12a5377ac9f9; no voters:
I20260927 18:39:32.350236 21004 leader_election.cc:288] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 18:39:32.350276 21008 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 18:39:32.350430 21008 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [term 2 LEADER]: Becoming Leader. State: Replica: 3e5894f2d08a4894b54e12a5377ac9f9, State: Running, Role: LEADER
I20260927 18:39:32.350521 21008 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5894f2d08a4894b54e12a5377ac9f9" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41019 } }
I20260927 18:39:32.350576 21004 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:32.350813 21008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e5894f2d08a4894b54e12a5377ac9f9. Latest consensus state: current_term: 2 leader_uuid: "3e5894f2d08a4894b54e12a5377ac9f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5894f2d08a4894b54e12a5377ac9f9" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41019 } } }
I20260927 18:39:32.350889 21008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:32.350819 21010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3e5894f2d08a4894b54e12a5377ac9f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5894f2d08a4894b54e12a5377ac9f9" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41019 } } }
I20260927 18:39:32.351097 21012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:39:32.351101 21010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:32.351759 21012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:39:32.351933 21012 catalog_manager.cc:1259] Loaded cluster ID: d0d18e9069274644bcfb942ee042fda7
I20260927 18:39:32.352167 21012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:39:32.353219 21012 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:39:32.353416 21012 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3e5894f2d08a4894b54e12a5377ac9f9: Loaded TSK: 0
I20260927 18:39:32.353710 21012 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 18:39:32.444294 21006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:32.444525 21006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:32.444550 21006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:32.445838 21006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:32.445883 21006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:32.445909 21006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:39:32.447449 21006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.193.keytab
--principal=oryx/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:40297
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=38363
--tserver_master_addrs=127.11.251.254:41019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:32.447711 21006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:32.447988 21006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:32.454525 21030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:32.454566 21033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:32.454524 21031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:32.454746 21006 server_base.cc:1034] running on GCE node
I20260927 18:39:32.455169 21006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:32.455457 21006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:32.456616 21006 hybrid_clock.cc:648] HybridClock initialized: now 1790534372456602 us; error 34 us; skew 500 ppm
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534372, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:32.459334 21006 init.cc:377] Logged in from keytab as oryx/127.11.251.193@KRBTEST.COM (short username oryx)
I20260927 18:39:32.459895 21006 webserver.cc:460] Webserver started at http://127.11.251.193:38363/ using document root <none> and password file <none>
I20260927 18:39:32.460152 21006 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:32.460214 21006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:32.461732 21006 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:32.462393 21040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:32.462591 21006 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:32.462695 21006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "70e9a9dd5e714667ac05c35e315e9e13"
format_stamp: "Formatted at 2026-09-27 18:39:31 on dist-test-slave-z47j"
server_key: "56d7d0820a89db5e09bbf5c776ea0dbd"
server_key_iv: "7bdefec336da42d54b3d74097be024ae"
server_key_version: "encryptionkey@0"
I20260927 18:39:32.463205 21006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:32.495224 21006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:32.498200 21006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:32.498358 21006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:32.498638 21006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:32.499017 21006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:32.499064 21006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:32.499099 21006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:32.499122 21006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:32.507126 21006 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:40297
I20260927 18:39:32.507191 21153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:40297 every 8 connection(s)
I20260927 18:39:32.507601 21006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:32.516952 21156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:32.508470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.193:52275)
0927 18:39:32.508676 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:32.508762 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:32.509348 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:32.509870 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:32.509875 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:32.510531 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:32.510821 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:32.510826 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.511967 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.511968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:32.512792 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:32.512794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.513414 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.513415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:32.513554 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:32.514279 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:32.514288 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:32.516703 (+ 2415us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":113,"threads_started":1}
W20260927 18:39:32.516987 21157 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:32.508545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:52275 (local address 127.11.251.254:41019)
0927 18:39:32.508798 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:32.508810 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:32.508862 (+ 52us) server_negotiation.cc:407] Connection header received
0927 18:39:32.509568 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:32.509599 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:32.509674 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:32.509830 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:32.510938 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.511856 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.512884 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.513316 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.513444 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:32.516833 (+ 3389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.193:52275: BlockingRecv error: recv got EOF from 127.11.251.193:52275 (error 108)
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":93,"threads_started":1}
W20260927 18:39:32.517319 21154 heartbeater.cc:644] Failed to heartbeat to 127.11.251.254:41019 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.251.254:41019: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
I20260927 18:39:32.517333 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 21006
I20260927 18:39:32.517706 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:42987
--local_ip_for_outbound_sockets=127.11.251.194
--tserver_master_addrs=127.11.251.254:41019
--webserver_port=43819
--webserver_interface=127.11.251.194
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.251.194
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:32.522533 21156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:32.517653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.193:59593)
0927 18:39:32.517748 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:32.517757 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:32.517833 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:32.518014 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:32.518016 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:32.518181 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:32.518246 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:32.518248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.518936 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.518936 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:32.519616 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:32.519618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.520026 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.520027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:32.520105 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:32.520574 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:32.520584 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:32.522423 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":82}
W20260927 18:39:32.522559 21157 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:32.517674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:59593 (local address 127.11.251.254:41019)
0927 18:39:32.517715 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:32.517717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:32.517893 (+ 176us) server_negotiation.cc:407] Connection header received
0927 18:39:32.517921 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:32.517922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:32.517935 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:32.517983 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:32.518359 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.518852 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.519720 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.519940 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.520026 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:32.522506 (+ 2480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.193:59593: BlockingRecv error: recv got EOF from 127.11.251.193:59593 (error 108)
Metrics: {"server-negotiator.queue_time_us":14,"thread_start_us":111,"threads_started":1}
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:32.527558 21156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:32.522805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.193:47677)
0927 18:39:32.522872 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:32.522884 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:32.522918 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:32.523044 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:32.523045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:32.523215 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:32.523257 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:32.523259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.523862 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.523863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:32.524774 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:32.524777 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.525118 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.525119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:32.525193 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:32.525619 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:32.525625 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:32.527473 (+ 1848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260927 18:39:32.527616 21157 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:32.522863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:47677 (local address 127.11.251.254:41019)
0927 18:39:32.522896 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:32.522898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:32.522923 (+ 25us) server_negotiation.cc:407] Connection header received
0927 18:39:32.522985 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:32.522986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:32.522995 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:32.523034 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:32.523331 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.523781 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.524865 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.525039 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.525101 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:32.527550 (+ 2449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.193:47677: BlockingRecv error: recv got EOF from 127.11.251.193:47677 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260927 18:39:32.527691 21154 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 18:39:32.610675 21159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:32.611073 21159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:32.611126 21159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:32.612555 21159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:32.612643 21159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:32.612699 21159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:39:32.614697 21159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.194.keytab
--principal=oryx/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:42987
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=43819
--tserver_master_addrs=127.11.251.254:41019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:32.615041 21159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:32.615406 21159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:32.621411 21165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:32.621389 21164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:32.621392 21167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:32.621421 21159 server_base.cc:1034] running on GCE node
I20260927 18:39:32.621879 21159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:32.622148 21159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:32.623308 21159 hybrid_clock.cc:648] HybridClock initialized: now 1790534372623294 us; error 50 us; skew 500 ppm
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534372, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:32.626076 21159 init.cc:377] Logged in from keytab as oryx/127.11.251.194@KRBTEST.COM (short username oryx)
I20260927 18:39:32.626528 21159 webserver.cc:460] Webserver started at http://127.11.251.194:43819/ using document root <none> and password file <none>
I20260927 18:39:32.626742 21159 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:32.626801 21159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:32.628172 21159 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:32.628793 21174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:32.629025 21159 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 18:39:32.629101 21159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "f0b554de5ce74c8a8f8b27298456cfe8"
format_stamp: "Formatted at 2026-09-27 18:39:31 on dist-test-slave-z47j"
server_key: "2a100dab274c2e3f8232cbcada71a4d9"
server_key_iv: "3b59c3bf67a72c95274fe64ac0fd15dd"
server_key_version: "encryptionkey@0"
I20260927 18:39:32.629496 21159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:32.654803 21159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:32.657373 21159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:32.657552 21159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:32.657883 21159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:32.658232 21159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:32.658267 21159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:32.658291 21159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:32.658334 21159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:32.664983 21159 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:42987
I20260927 18:39:32.665079 21287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:42987 every 8 connection(s)
I20260927 18:39:32.665442 21159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
W20260927 18:39:32.673494 21157 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:32.666255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:37473 (local address 127.11.251.254:41019)
0927 18:39:32.666331 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:32.666334 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:32.666594 (+ 260us) server_negotiation.cc:407] Connection header received
0927 18:39:32.667223 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:32.667224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:32.667245 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:32.667298 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:32.668408 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.668966 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.670068 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.670302 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.670390 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:32.673430 (+ 3040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.194:37473: BlockingRecv error: recv got EOF from 127.11.251.194:37473 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:39:32.673556 21290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:32.666260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.194:37473)
0927 18:39:32.666443 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:32.666488 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:32.667102 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:32.667392 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:32.667396 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:32.668079 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:32.668304 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:32.668310 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.669066 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.669067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:32.669971 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:32.669974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.670399 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.670400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:32.670527 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:32.671081 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:32.671090 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:32.673342 (+ 2252us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
W20260927 18:39:32.673832 21288 heartbeater.cc:644] Failed to heartbeat to 127.11.251.254:41019 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.251.254:41019: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
I20260927 18:39:32.673904 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 21159
I20260927 18:39:32.674273 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:37135
--local_ip_for_outbound_sockets=127.11.251.195
--tserver_master_addrs=127.11.251.254:41019
--webserver_port=37447
--webserver_interface=127.11.251.195
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.251.195
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:32.678853 21290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:32.674095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.194:46343)
0927 18:39:32.674233 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:32.674243 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:32.674297 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:32.674509 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:32.674510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:32.674624 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:32.674674 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:32.674675 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.675546 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.675546 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:32.676038 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:32.676039 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.676407 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.676407 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:32.676480 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:32.676926 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:32.676930 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:32.678768 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122}
W20260927 18:39:32.678913 21157 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:32.674184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:46343 (local address 127.11.251.254:41019)
0927 18:39:32.674253 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:32.674256 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:32.674285 (+ 29us) server_negotiation.cc:407] Connection header received
0927 18:39:32.674379 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:32.674380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:32.674396 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:32.674457 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:32.674752 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.675471 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.676124 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.676332 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.676411 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:32.678842 (+ 2431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.194:46343: BlockingRecv error: recv got EOF from 127.11.251.194:46343 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:32.683409 21290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:32.679158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.194:56347)
0927 18:39:32.679245 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:32.679254 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:32.679309 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:32.679449 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:32.679449 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:32.679574 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:32.679599 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:32.679600 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.680236 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.680237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:32.680724 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:32.680726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.681076 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.681077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:32.681130 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:32.681541 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:32.681545 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:32.683330 (+ 1785us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":74}
W20260927 18:39:32.683465 21157 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:32.679254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:56347 (local address 127.11.251.254:41019)
0927 18:39:32.679313 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:32.679315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:32.679324 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:39:32.679366 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:32.679367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:32.679377 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:32.679447 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:32.679672 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.680129 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.680805 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.680992 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.681076 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:32.683398 (+ 2322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.194:56347: BlockingRecv error: recv got EOF from 127.11.251.194:56347 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260927 18:39:32.683833 21288 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 18:39:32.766743 21291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:32.767048 21291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:32.767076 21291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:32.768411 21291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:32.768456 21291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:32.768486 21291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:39:32.770186 21291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/krb5kdc/oryx_127.11.251.195.keytab
--principal=oryx/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:37135
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=37447
--tserver_master_addrs=127.11.251.254:41019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:32.770483 21291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:32.770753 21291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:39:32.776453 21291 server_base.cc:1034] running on GCE node
W20260927 18:39:32.776420 21296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:32.776420 21299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:32.776412 21297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:32.776871 21291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:32.777108 21291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:32.778256 21291 hybrid_clock.cc:648] HybridClock initialized: now 1790534372778245 us; error 32 us; skew 500 ppm
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534372, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:32.781283 21291 init.cc:377] Logged in from keytab as oryx/127.11.251.195@KRBTEST.COM (short username oryx)
I20260927 18:39:32.781885 21291 webserver.cc:460] Webserver started at http://127.11.251.195:37447/ using document root <none> and password file <none>
I20260927 18:39:32.782111 21291 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:32.782172 21291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:32.783792 21291 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:39:32.784497 21306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:32.784716 21291 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:32.784832 21291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "dacd9db039614d0bb455e4ca9d9151c9"
format_stamp: "Formatted at 2026-09-27 18:39:32 on dist-test-slave-z47j"
server_key: "0a5bdae0e32a6e33b6ad84b759c2acc9"
server_key_iv: "548995e0523fc27128c25b3c29f0b74c"
server_key_version: "encryptionkey@0"
I20260927 18:39:32.785269 21291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:32.806036 21291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:32.808296 21291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:32.808482 21291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:32.808769 21291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:32.809139 21291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:32.809195 21291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:32.809240 21291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:32.809284 21291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:32.816617 21291 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:37135
I20260927 18:39:32.816674 21419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:37135 every 8 connection(s)
I20260927 18:39:32.817287 21291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:39:32.820823 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 21291
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
W20260927 18:39:32.827188 21157 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:32.818262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:33631 (local address 127.11.251.254:41019)
0927 18:39:32.818386 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:32.818388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:32.818572 (+ 184us) server_negotiation.cc:407] Connection header received
0927 18:39:32.819223 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:32.819226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:32.819241 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:32.819295 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:32.820354 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.821122 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.822833 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.823143 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.823210 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:32.827129 (+ 3919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.195:33631: BlockingRecv error: recv got EOF from 127.11.251.195:33631 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260927 18:39:32.827325 21422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:32.818176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.195:33631)
0927 18:39:32.818419 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:32.818476 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:32.819090 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:32.819369 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:32.819374 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:32.820018 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:32.820241 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:32.820246 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.821413 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.821414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:32.822753 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:32.822755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.823235 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.823236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:32.823384 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:32.823996 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:32.824007 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:32.827049 (+ 3042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":117,"threads_started":1}
W20260927 18:39:32.827654 21420 heartbeater.cc:644] Failed to heartbeat to 127.11.251.254:41019 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.251.254:41019: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
I20260927 18:39:32.829674 21158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:32.821402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32854 (local address 127.11.251.254:41019)
0927 18:39:32.821454 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:32.821455 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:32.821464 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:39:32.821525 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:32.821526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:32.821534 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:32.821672 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:32.822450 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.823128 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.824199 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.824402 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.824465 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:32.825985 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:32.825991 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:32.825991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:32.826014 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:32.828108 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:32.828518 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:32.828522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:32.828522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:32.828663 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:32.828905 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:32.828906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:32.828906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:32.829166 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:32.829310 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:32.829443 (+ 133us) server_negotiation.cc:299] Negotiation successful
0927 18:39:32.829556 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:32.832674 21422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:32.827935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.195:34595)
0927 18:39:32.828041 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:32.828053 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:32.828113 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:32.828311 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:32.828312 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:32.828437 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:32.828523 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:32.828525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.829342 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.829343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:32.830070 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:32.830072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.830441 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.830442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:32.830515 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:32.830904 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:32.830909 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:32.832623 (+ 1714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":86}
W20260927 18:39:32.832801 21157 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:32.827989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:34595 (local address 127.11.251.254:41019)
0927 18:39:32.828029 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:32.828031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:32.828108 (+ 77us) server_negotiation.cc:407] Connection header received
0927 18:39:32.828203 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:32.828204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:32.828215 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:32.828282 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:32.828615 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.829241 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.830145 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.830362 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.830431 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:32.832706 (+ 2275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.195:34595: BlockingRecv error: recv got EOF from 127.11.251.195:34595 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:32 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:32.837175 21422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:32.833004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.195:56629)
0927 18:39:32.833062 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:32.833075 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:32.833122 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:32.833268 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:32.833269 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:32.833372 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:32.833429 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:32.833430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.834147 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.834148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:32.834619 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:32.834621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.834990 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.834991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:32.835036 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:32.835447 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:32.835451 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:32.837122 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260927 18:39:32.837393 21157 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:32.833070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:56629 (local address 127.11.251.254:41019)
0927 18:39:32.833116 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:32.833118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:32.833127 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:39:32.833187 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:32.833188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:32.833197 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:32.833257 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:32.833510 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.834068 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.834695 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:32.834903 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:32.834960 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:32.837304 (+ 2344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.195:56629: BlockingRecv error: recv got EOF from 127.11.251.195:56629 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260927 18:39:32.837436 21420 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 27 18:39:33 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:33 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:33.533798 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:33.528154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.193:45121)
0927 18:39:33.528369 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:33.528386 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:33.528465 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:33.528644 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:33.528645 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:33.528839 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:33.528898 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:33.528900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:33.529791 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:33.529792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:33.530382 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:33.530384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:33.530787 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:33.530788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:33.530838 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:33.531379 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:33.531384 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:33.533726 (+ 2342us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":96,"threads_started":1}
W20260927 18:39:33.533999 21425 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:33.528224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:45121 (local address 127.11.251.254:41019)
0927 18:39:33.528437 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:33.528440 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:33.528450 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:39:33.528529 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:33.528530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:33.528542 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:33.528613 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:33.529010 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:33.529705 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:33.530469 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:33.530710 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:33.530775 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:33.533887 (+ 3112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.193:45121: BlockingRecv error: recv got EOF from 127.11.251.193:45121 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":137,"threads_started":1}
Sep 27 18:39:33 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:33 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:33.689792 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:33.684345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.194:53277)
0927 18:39:33.684526 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:33.684540 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:33.684633 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:33.684794 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:33.684795 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:33.684978 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:33.685036 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:33.685038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:33.685877 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:33.685878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:33.686458 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:33.686459 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:33.686860 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:33.686860 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:33.686901 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:33.687401 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:33.687406 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:33.689655 (+ 2249us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":76,"threads_started":1}
W20260927 18:39:33.689811 21425 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:33.684451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:53277 (local address 127.11.251.254:41019)
0927 18:39:33.684523 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:33.684526 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:33.684639 (+ 113us) server_negotiation.cc:407] Connection header received
0927 18:39:33.684694 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:33.684695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:33.684712 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:33.684771 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:33.685125 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:33.685775 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:33.686546 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:33.686774 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:33.686839 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:33.689734 (+ 2895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.194:53277: BlockingRecv error: recv got EOF from 127.11.251.194:53277 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 27 18:39:33 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:33 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:33.843262 21427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:33.837915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.195:42137)
0927 18:39:33.838129 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:33.838143 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:33.838251 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:33.838441 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:33.838441 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:33.838668 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:33.838729 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:33.838730 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:33.839488 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:33.839490 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:33.840091 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:33.840092 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:33.840498 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:33.840499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:33.840560 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:33.841134 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:33.841139 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:33.843121 (+ 1982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":106,"threads_started":1}
W20260927 18:39:33.843312 21425 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:33.838027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:42137 (local address 127.11.251.254:41019)
0927 18:39:33.838125 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:33.838128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:33.838226 (+ 98us) server_negotiation.cc:407] Connection header received
0927 18:39:33.838336 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:33.838338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:33.838357 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:33.838431 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:33.838825 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:33.839391 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:33.840183 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:33.840392 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:33.840500 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:33.843216 (+ 2716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.195:42137: BlockingRecv error: recv got EOF from 127.11.251.195:42137 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 27 18:39:34 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:34 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:34.539736 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:34.534369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.193:48941)
0927 18:39:34.534547 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:34.534559 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:34.534631 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:34.534837 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:34.534838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:34.535044 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:34.535118 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:34.535120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:34.535974 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:34.535975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:34.536593 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:34.536596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:34.536993 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:34.536994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:34.537042 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:34.537629 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:34.537634 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:34.539639 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":101,"threads_started":1}
W20260927 18:39:34.539858 21429 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:34.534509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:48941 (local address 127.11.251.254:41019)
0927 18:39:34.534673 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:34.534675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:34.534684 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:39:34.534720 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:34.534720 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:34.534731 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:34.534808 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:34.535227 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:34.535819 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:34.536700 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:34.536908 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:34.536975 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:34.539765 (+ 2790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.193:48941: BlockingRecv error: recv got EOF from 127.11.251.193:48941 (error 108)
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":93,"threads_started":1}
Sep 27 18:39:34 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:34 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:34.696103 21430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:34.690370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.194:44311)
0927 18:39:34.690589 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:34.690608 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:34.690760 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:34.690967 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:34.690967 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:34.691188 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:34.691258 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:34.691259 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:34.692002 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:34.692003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:34.692601 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:34.692602 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:34.692993 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:34.692994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:34.693052 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:34.693798 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:34.693803 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:34.695990 (+ 2187us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":101,"threads_started":1}
W20260927 18:39:34.696168 21429 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:34.690528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:44311 (local address 127.11.251.254:41019)
0927 18:39:34.690636 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:34.690640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:34.690725 (+ 85us) server_negotiation.cc:407] Connection header received
0927 18:39:34.690853 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:34.690854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:34.690876 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:34.690942 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:34.691356 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:34.691910 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:34.692692 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:34.692901 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:34.692983 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:34.696096 (+ 3113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.194:44311: BlockingRecv error: recv got EOF from 127.11.251.194:44311 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 27 18:39:34 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:34 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:34.849850 21431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:34.843952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.195:56649)
0927 18:39:34.844139 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:34.844156 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:34.844355 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:34.844548 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:34.844549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:34.844778 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:34.844840 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:34.844842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:34.845679 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:34.845680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:34.846227 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:34.846228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:34.846689 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:34.846691 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:34.846967 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:34.847601 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:34.847605 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:34.849740 (+ 2135us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":60,"threads_started":1}
W20260927 18:39:34.849895 21429 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:34.844257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:56649 (local address 127.11.251.254:41019)
0927 18:39:34.844367 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:34.844370 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:34.844401 (+ 31us) server_negotiation.cc:407] Connection header received
0927 18:39:34.844436 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:34.844437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:34.844461 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:34.844540 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:34.844934 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:34.845497 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:34.846317 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:34.846554 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:34.846638 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:34.849824 (+ 3186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.195:56649: BlockingRecv error: recv got EOF from 127.11.251.195:56649 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 27 18:39:35 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:35 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:35.550824 21432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:35.540507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.193:40637)
0927 18:39:35.540836 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:35.540850 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:35.540947 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:35.541178 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:35.541179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:35.541378 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:35.541451 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:35.541453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:35.542286 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:35.542286 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:35.542884 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:35.542885 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:35.543290 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:35.543291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:35.543354 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:35.548279 (+ 4925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:35.548284 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:35.550756 (+ 2472us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":164,"threads_started":1}
W20260927 18:39:35.551070 21433 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:35.540578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:40637 (local address 127.11.251.254:41019)
0927 18:39:35.540799 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:35.540801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:35.540927 (+ 126us) server_negotiation.cc:407] Connection header received
0927 18:39:35.541029 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:35.541030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:35.541043 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:35.541133 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:35.541549 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:35.542206 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:35.542971 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:35.543169 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:35.543228 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:35.550923 (+ 7695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.193:40637: BlockingRecv error: recv got EOF from 127.11.251.193:40637 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":125,"threads_started":1}
Sep 27 18:39:35 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:35 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:35.702169 21434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:35.696767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.194:50759)
0927 18:39:35.696931 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:35.696943 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:35.697005 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:35.697183 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:35.697184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:35.697379 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:35.697449 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:35.697451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:35.698175 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:35.698176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:35.698828 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:35.698829 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:35.699193 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:35.699194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:35.699236 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:35.699814 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:35.699819 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:35.702093 (+ 2274us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
W20260927 18:39:35.702280 21433 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:35.696880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:50759 (local address 127.11.251.254:41019)
0927 18:39:35.696968 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:35.696970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:35.697000 (+ 30us) server_negotiation.cc:407] Connection header received
0927 18:39:35.697069 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:35.697070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:35.697082 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:35.697151 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:35.697537 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:35.698087 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:35.698907 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:35.699121 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:35.699180 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:35.702199 (+ 3019us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.194:50759: BlockingRecv error: recv got EOF from 127.11.251.194:50759 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 27 18:39:35 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:35 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:35.856278 21435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:35.850474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.195:39135)
0927 18:39:35.850686 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:35.850702 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:35.850811 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:35.850975 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:35.850976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:35.851181 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:35.851286 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:35.851289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:35.852122 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:35.852123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:35.852800 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:35.852802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:35.853160 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:35.853161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:35.853211 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:35.853939 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:35.853944 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:35.856166 (+ 2222us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":97,"threads_started":1}
W20260927 18:39:35.856312 21433 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:35.850592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:39135 (local address 127.11.251.254:41019)
0927 18:39:35.850654 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:35.850656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:35.850807 (+ 151us) server_negotiation.cc:407] Connection header received
0927 18:39:35.850877 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:35.850878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:35.850893 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:35.850939 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:35.851406 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:35.851983 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:35.852888 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:35.853080 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:35.853134 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:35.856250 (+ 3116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.195:39135: BlockingRecv error: recv got EOF from 127.11.251.195:39135 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 27 18:39:36 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:36 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:36.556813 21436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:36.551457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.193:47977)
0927 18:39:36.551678 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:36.551689 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:36.551764 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:36.551948 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:36.551949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:36.552145 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:36.552233 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:36.552235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:36.552909 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:36.552910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:36.553482 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:36.553483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:36.553942 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:36.553944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:36.554010 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:36.554538 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:36.554542 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:36.556741 (+ 2199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":100,"threads_started":1}
W20260927 18:39:36.557009 21437 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:36.551504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:47977 (local address 127.11.251.254:41019)
0927 18:39:36.551654 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:36.551656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:36.551748 (+ 92us) server_negotiation.cc:407] Connection header received
0927 18:39:36.551828 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:36.551829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:36.551843 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:36.551922 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:36.552288 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:36.552829 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:36.553613 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:36.553817 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:36.553898 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:36.556864 (+ 2966us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.193:47977: BlockingRecv error: recv got EOF from 127.11.251.193:47977 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
Sep 27 18:39:36 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:36 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:36.707808 21438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:36.702855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.194:58259)
0927 18:39:36.703043 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:36.703057 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:36.703148 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:36.703318 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:36.703319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:36.703514 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:36.703583 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:36.703585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:36.704246 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:36.704247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:36.704853 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:36.704855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:36.705237 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:36.705238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:36.705295 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:36.705802 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:36.705807 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:36.707732 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":88,"threads_started":1}
W20260927 18:39:36.707937 21437 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:36.702954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:58259 (local address 127.11.251.254:41019)
0927 18:39:36.703036 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:36.703037 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:36.703150 (+ 113us) server_negotiation.cc:407] Connection header received
0927 18:39:36.703216 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:36.703217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:36.703231 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:36.703293 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:36.703655 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:36.704157 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:36.704934 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:36.705124 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:36.705182 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:36.707835 (+ 2653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.194:58259: BlockingRecv error: recv got EOF from 127.11.251.194:58259 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 27 18:39:36 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:36 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:36.861625 21439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:36.856871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.195:51073)
0927 18:39:36.857064 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:36.857077 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:36.857170 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:36.857317 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:36.857318 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:36.857501 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:36.857549 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:36.857551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:36.858257 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:36.858258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:36.858790 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:36.858791 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:36.859178 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:36.859179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:36.859237 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:36.859738 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:36.859743 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:36.861507 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":109,"threads_started":1}
W20260927 18:39:36.861672 21437 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:36.856952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:51073 (local address 127.11.251.254:41019)
0927 18:39:36.857033 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:36.857036 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:36.857148 (+ 112us) server_negotiation.cc:407] Connection header received
0927 18:39:36.857234 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:36.857235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:36.857250 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:36.857297 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:36.857674 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:36.858176 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:36.858887 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:36.859094 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:36.859176 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:36.861606 (+ 2430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.195:51073: BlockingRecv error: recv got EOF from 127.11.251.195:51073 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 27 18:39:37 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:37 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:37.563133 21440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:37.557367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.193:45591)
0927 18:39:37.557555 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:37.557565 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:37.557672 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:37.557891 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:37.557891 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:37.558063 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:37.558125 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:37.558127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:37.558872 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:37.558873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:37.559428 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:37.559430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:37.560064 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:37.560065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:37.560123 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:37.560701 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:37.560706 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:37.563055 (+ 2349us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":85,"threads_started":1}
W20260927 18:39:37.563330 21441 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:37.557473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:45591 (local address 127.11.251.254:41019)
0927 18:39:37.557641 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:37.557644 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:37.557662 (+ 18us) server_negotiation.cc:407] Connection header received
0927 18:39:37.557751 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:37.557752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:37.557789 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:37.557868 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:37.558199 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:37.558784 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:37.559594 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:37.559932 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:37.560019 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:37.563211 (+ 3192us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.193:45591: BlockingRecv error: recv got EOF from 127.11.251.193:45591 (error 108)
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":95,"threads_started":1}
Sep 27 18:39:37 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:37 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:37.714178 21442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:37.708517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.194:55817)
0927 18:39:37.708719 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:37.708732 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:37.708810 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:37.709005 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:37.709006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:37.709200 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:37.709268 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:37.709271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:37.710093 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:37.710094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:37.710717 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:37.710719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:37.711104 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:37.711104 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:37.711157 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:37.711735 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:37.711745 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:37.714046 (+ 2301us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":109,"threads_started":1}
W20260927 18:39:37.714327 21441 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:37.708686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:55817 (local address 127.11.251.254:41019)
0927 18:39:37.708775 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:37.708777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:37.708795 (+ 18us) server_negotiation.cc:407] Connection header received
0927 18:39:37.708893 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:37.708894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:37.708906 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:37.708978 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:37.709371 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:37.710001 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:37.710806 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:37.711021 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:37.711086 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:37.714234 (+ 3148us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.194:55817: BlockingRecv error: recv got EOF from 127.11.251.194:55817 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 27 18:39:37 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:37 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:37.867647 21443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:37.862265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.195:45443)
0927 18:39:37.862419 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:37.862434 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:37.862505 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:37.862709 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:37.862710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:37.862897 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:37.862953 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:37.862954 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:37.863716 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:37.863717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:37.864276 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:37.864278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:37.864684 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:37.864685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:37.864725 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:37.865287 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:37.865291 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:37.867578 (+ 2287us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":148,"threads_started":1}
W20260927 18:39:37.867851 21441 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:37.862414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:45443 (local address 127.11.251.254:41019)
0927 18:39:37.862512 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:37.862514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:37.862529 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:39:37.862560 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:37.862561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:37.862580 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:37.862687 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:37.863065 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:37.863622 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:37.864384 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:37.864602 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:37.864678 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:37.867766 (+ 3088us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.195:45443: BlockingRecv error: recv got EOF from 127.11.251.195:45443 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 27 18:39:38 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:38 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:38.569428 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:38.563750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.193:36657)
0927 18:39:38.563957 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:38.563971 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:38.564064 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:38.564295 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:38.564296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:38.564501 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:38.564570 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:38.564571 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:38.565337 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:38.565338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:38.566032 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:38.566034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:38.566464 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:38.566465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:38.566509 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:38.567089 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:38.567094 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:38.569322 (+ 2228us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":105,"threads_started":1}
W20260927 18:39:38.569518 21445 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:38.563930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:36657 (local address 127.11.251.254:41019)
0927 18:39:38.564102 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:38.564104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:38.564116 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:39:38.564143 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:38.564144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:38.564156 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:38.564233 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:38.564668 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:38.565255 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:38.566123 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:38.566340 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:38.566440 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:38.569437 (+ 2997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.193:36657: BlockingRecv error: recv got EOF from 127.11.251.193:36657 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
Sep 27 18:39:38 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:38 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:38.720508 21446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:38.714811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.194:39075)
0927 18:39:38.714986 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:38.714998 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:38.715075 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:38.715375 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:38.715377 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:38.715583 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:38.715683 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:38.715686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:38.716576 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:38.716577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:38.717308 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:38.717310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:38.717769 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:38.717771 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:38.717823 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:38.718326 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:38.718331 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:38.720410 (+ 2079us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":107,"threads_started":1}
W20260927 18:39:38.720584 21445 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:38.714903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:39075 (local address 127.11.251.254:41019)
0927 18:39:38.714977 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:38.714979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:38.715076 (+ 97us) server_negotiation.cc:407] Connection header received
0927 18:39:38.715220 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:38.715221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:38.715233 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:38.715313 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:38.715779 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:38.716420 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:38.717396 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:38.717646 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:38.717713 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:38.720515 (+ 2802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.194:39075: BlockingRecv error: recv got EOF from 127.11.251.194:39075 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 27 18:39:38 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:38 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:38.873483 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:38.868326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.195:34867)
0927 18:39:38.868532 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:38.868550 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:38.868647 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:38.868805 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:38.868806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:38.869010 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:38.869055 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:38.869057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:38.869793 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:38.869794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:38.870307 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:38.870308 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:38.870696 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:38.870697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:38.870753 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:38.871252 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:38.871265 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:38.873411 (+ 2146us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":90,"threads_started":1}
W20260927 18:39:38.873667 21445 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:38.868386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:34867 (local address 127.11.251.254:41019)
0927 18:39:38.868481 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:38.868484 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:38.868622 (+ 138us) server_negotiation.cc:407] Connection header received
0927 18:39:38.868703 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:38.868704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:38.868721 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:38.868796 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:38.869137 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:38.869714 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:38.870422 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:38.870615 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:38.870695 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:38.873518 (+ 2823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.195:34867: BlockingRecv error: recv got EOF from 127.11.251.195:34867 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Sep 27 18:39:39 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:39 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:39.575377 21448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:39.570046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.193:59197)
0927 18:39:39.570239 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:39.570252 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:39.570318 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:39.570494 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:39.570495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:39.570665 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:39.570726 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:39.570728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:39.571476 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:39.571477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:39.572143 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:39.572146 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:39.572557 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:39.572558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:39.572610 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:39.573159 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:39.573164 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:39.575310 (+ 2146us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":110,"threads_started":1}
W20260927 18:39:39.575592 21449 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:39.570167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:59197 (local address 127.11.251.254:41019)
0927 18:39:39.570328 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:39.570330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:39.570339 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:39:39.570378 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:39.570379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:39.570391 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:39.570460 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:39.570814 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:39.571372 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:39.572238 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:39.572445 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:39.572505 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:39.575493 (+ 2988us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.193:59197: BlockingRecv error: recv got EOF from 127.11.251.193:59197 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
Sep 27 18:39:39 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:39 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:39.726554 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:39.721081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.194:36071)
0927 18:39:39.721249 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:39.721260 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:39.721319 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:39.721550 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:39.721550 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:39.721735 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:39.721795 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:39.721797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:39.722673 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:39.722673 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:39.723301 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:39.723303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:39.723686 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:39.723687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:39.723731 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:39.724283 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:39.724288 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:39.726441 (+ 2153us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":92,"threads_started":1}
W20260927 18:39:39.726598 21449 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:39.721266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:36071 (local address 127.11.251.254:41019)
0927 18:39:39.721347 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:39.721349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:39.721359 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:39:39.721387 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:39.721387 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:39.721441 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:39.721538 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:39.721878 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:39.722575 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:39.723391 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:39.723610 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:39.723673 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:39.726530 (+ 2857us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.194:36071: BlockingRecv error: recv got EOF from 127.11.251.194:36071 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 27 18:39:39 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:39 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:39.879392 21451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:39.874054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.195:33815)
0927 18:39:39.874424 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:39.874436 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:39.874559 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:39.874760 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:39.874761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:39.874986 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:39.875046 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:39.875048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:39.875769 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:39.875770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:39.876344 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:39.876345 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:39.876728 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:39.876729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:39.876782 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:39.877373 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:39.877398 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:39.879275 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":132,"threads_started":1}
W20260927 18:39:39.879434 21449 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:39.874178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:33815 (local address 127.11.251.254:41019)
0927 18:39:39.874291 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:39.874293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:39.874534 (+ 241us) server_negotiation.cc:407] Connection header received
0927 18:39:39.874654 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:39.874656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:39.874680 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:39.874752 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:39.875143 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:39.875686 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:39.876434 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:39.876639 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:39.876731 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:39.879365 (+ 2634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.195:33815: BlockingRecv error: recv got EOF from 127.11.251.195:33815 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 27 18:39:40 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:40 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:40.588290 21452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:40.576129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.193:39423)
0927 18:39:40.576309 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:40.576330 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:40.576395 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:40.576748 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:40.576750 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:40.576930 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:40.576986 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:40.576988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:40.577827 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:40.577828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:40.578364 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:40.578365 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:40.578806 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:40.578807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:40.578896 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:40.585901 (+ 7005us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:40.585908 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:40.588201 (+ 2293us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":112,"threads_started":1}
W20260927 18:39:40.588492 21453 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:40.576394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:39423 (local address 127.11.251.254:41019)
0927 18:39:40.576555 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:40.576557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:40.576566 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:39:40.576596 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:40.576597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:40.576608 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:40.576681 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:40.577102 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:40.577718 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:40.578492 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:40.578691 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:40.578759 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:40.588385 (+ 9626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.193:39423: BlockingRecv error: recv got EOF from 127.11.251.193:39423 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":97,"threads_started":1}
Sep 27 18:39:40 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:40 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:40.732784 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:40.727203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.194:49157)
0927 18:39:40.727368 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:40.727387 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:40.727459 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:40.727670 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:40.727672 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:40.727879 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:40.727950 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:40.727952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:40.728812 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:40.728813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:40.729389 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:40.729391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:40.730006 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:40.730007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:40.730067 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:40.730565 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:40.730570 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:40.732714 (+ 2144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
W20260927 18:39:40.732935 21453 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:40.727300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:49157 (local address 127.11.251.254:41019)
0927 18:39:40.727377 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:40.727379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:40.727463 (+ 84us) server_negotiation.cc:407] Connection header received
0927 18:39:40.727515 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:40.727516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:40.727529 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:40.727607 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:40.728032 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:40.728710 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:40.729520 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:40.729916 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:40.730006 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:40.732833 (+ 2827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.194:49157: BlockingRecv error: recv got EOF from 127.11.251.194:49157 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 27 18:39:40 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:40 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:40.885393 21455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:40.880011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.195:57447)
0927 18:39:40.880264 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:40.880276 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:40.880351 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:40.880509 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:40.880510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:40.880705 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:40.880782 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:40.880784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:40.881486 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:40.881487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:40.882114 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:40.882116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:40.882457 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:40.882458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:40.882505 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:40.883004 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:40.883016 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:40.885327 (+ 2311us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":128,"threads_started":1}
W20260927 18:39:40.885540 21453 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:40.880123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:57447 (local address 127.11.251.254:41019)
0927 18:39:40.880213 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:40.880215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:40.880354 (+ 139us) server_negotiation.cc:407] Connection header received
0927 18:39:40.880406 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:40.880407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:40.880420 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:40.880484 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:40.880861 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:40.881375 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:40.882193 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:40.882374 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:40.882432 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:40.885441 (+ 3009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.195:57447: BlockingRecv error: recv got EOF from 127.11.251.195:57447 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 27 18:39:41 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:41 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:41.594709 21456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:41.588820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.193:38893)
0927 18:39:41.588976 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:41.588988 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:41.589048 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:41.589330 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:41.589331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:41.589540 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:41.589625 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:41.589627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:41.590366 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:41.590367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:41.590943 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:41.590945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:41.591346 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:41.591347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:41.591405 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:41.591969 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:41.591974 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:41.594598 (+ 2624us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":106,"threads_started":1}
W20260927 18:39:41.594754 21457 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:41.588960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:38893 (local address 127.11.251.254:41019)
0927 18:39:41.589121 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:41.589123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:41.589133 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:39:41.589182 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:41.589183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:41.589196 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:41.589282 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:41.589710 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:41.590278 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:41.591027 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:41.591240 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:41.591302 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:41.594677 (+ 3375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.193:38893: BlockingRecv error: recv got EOF from 127.11.251.193:38893 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":91,"threads_started":1}
Sep 27 18:39:41 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:41 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:41.739312 21458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:41.733469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.194:49065)
0927 18:39:41.733638 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:41.733651 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:41.733731 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:41.733965 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:41.733967 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:41.734191 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:41.734255 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:41.734257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:41.735132 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:41.735133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:41.735720 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:41.735722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:41.736172 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:41.736173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:41.736256 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:41.736928 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:41.736935 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:41.739213 (+ 2278us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":141,"threads_started":1}
W20260927 18:39:41.739523 21457 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:41.733568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:49065 (local address 127.11.251.254:41019)
0927 18:39:41.733696 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:41.733698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:41.733718 (+ 20us) server_negotiation.cc:407] Connection header received
0927 18:39:41.733823 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:41.733824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:41.733836 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:41.733911 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:41.734374 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:41.735045 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:41.735826 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:41.736077 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:41.736168 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:41.739351 (+ 3183us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.194:49065: BlockingRecv error: recv got EOF from 127.11.251.194:49065 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 27 18:39:41 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:41 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:41.891170 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:41.886021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.195:40151)
0927 18:39:41.886171 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:41.886185 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:41.886256 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:41.886499 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:41.886500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:41.886685 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:41.886732 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:41.886734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:41.887421 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:41.887422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:41.887999 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:41.888000 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:41.888442 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:41.888443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:41.888505 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:41.889051 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:41.889055 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:41.891095 (+ 2040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
W20260927 18:39:41.891361 21457 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:41.886150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:40151 (local address 127.11.251.254:41019)
0927 18:39:41.886237 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:41.886239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:41.886252 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:39:41.886320 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:41.886321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:41.886345 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:41.886486 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:41.886824 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:41.887321 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:41.888085 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:41.888349 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:41.888441 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:41.891247 (+ 2806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.195:40151: BlockingRecv error: recv got EOF from 127.11.251.195:40151 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 27 18:39:42 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:42 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:42.600481 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:42.595333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.193:44187)
0927 18:39:42.595536 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:42.595548 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:42.595615 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:42.595794 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:42.595795 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:42.595971 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:42.596027 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:42.596028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:42.597148 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:42.597149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:42.597767 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:42.597769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:42.598163 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:42.598163 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:42.598203 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:42.598718 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:42.598723 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:42.600419 (+ 1696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":122,"threads_started":1}
W20260927 18:39:42.600674 21461 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:42.595459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:44187 (local address 127.11.251.254:41019)
0927 18:39:42.595618 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:42.595621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:42.595637 (+ 16us) server_negotiation.cc:407] Connection header received
0927 18:39:42.595671 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:42.595673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:42.595689 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:42.595768 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:42.596138 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:42.597041 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:42.597852 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:42.598077 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:42.598144 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:42.600550 (+ 2406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.193:44187: BlockingRecv error: recv got EOF from 127.11.251.193:44187 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
Sep 27 18:39:42 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:42 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:42.744789 21462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:42.739935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.194:45309)
0927 18:39:42.740086 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:42.740101 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:42.740198 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:42.740375 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:42.740376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:42.740578 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:42.740631 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:42.740634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:42.741345 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:42.741347 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:42.741962 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:42.741964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:42.742347 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:42.742348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:42.742420 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:42.742922 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:42.742927 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:42.744685 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
W20260927 18:39:42.744838 21461 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:42.740007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:45309 (local address 127.11.251.254:41019)
0927 18:39:42.740075 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:42.740077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:42.740190 (+ 113us) server_negotiation.cc:407] Connection header received
0927 18:39:42.740270 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:42.740272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:42.740288 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:42.740366 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:42.740716 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:42.741246 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:42.742053 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:42.742258 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:42.742346 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:42.744766 (+ 2420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.194:45309: BlockingRecv error: recv got EOF from 127.11.251.194:45309 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 27 18:39:42 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:42 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:42.896982 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:42.891766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.195:58669)
0927 18:39:42.891958 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:42.891974 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:42.892068 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:42.892234 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:42.892235 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:42.892446 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:42.892512 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:42.892514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:42.893162 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:42.893162 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:42.893853 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:42.893861 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:42.894226 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:42.894226 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:42.894268 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:42.894959 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:42.894965 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:42.896917 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":98,"threads_started":1}
W20260927 18:39:42.897130 21461 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:42.891934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:58669 (local address 127.11.251.254:41019)
0927 18:39:42.892017 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:42.892020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:42.892054 (+ 34us) server_negotiation.cc:407] Connection header received
0927 18:39:42.892138 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:42.892139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:42.892154 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:42.892211 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:42.892585 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:42.893092 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:42.893941 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:42.894153 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:42.894217 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:42.897031 (+ 2814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.195:58669: BlockingRecv error: recv got EOF from 127.11.251.195:58669 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 27 18:39:43 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:43 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:43.606149 21464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:43.601127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.193:39759)
0927 18:39:43.601297 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:43.601313 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:43.601399 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:43.601676 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:43.601677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:43.601918 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:43.601981 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:43.601983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:43.602653 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:43.602654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:43.603242 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:43.603244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:43.603607 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:43.603608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:43.603651 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:43.604156 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:43.604161 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:43.606071 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":109,"threads_started":1}
W20260927 18:39:43.606282 21465 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:43.601178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:39759 (local address 127.11.251.254:41019)
0927 18:39:43.601336 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:43.601338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:43.601381 (+ 43us) server_negotiation.cc:407] Connection header received
0927 18:39:43.601477 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:43.601478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:43.601494 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:43.601658 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:43.602060 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:43.602579 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:43.603330 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:43.603526 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:43.603606 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:43.606177 (+ 2571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.193:39759: BlockingRecv error: recv got EOF from 127.11.251.193:39759 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":77,"threads_started":1}
Sep 27 18:39:43 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:43 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:43.750864 21466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:43.745367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.194:56489)
0927 18:39:43.745597 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:43.745631 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:43.745741 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:43.745899 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:43.745900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:43.746128 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:43.746188 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:43.746190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:43.746940 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:43.746941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:43.747513 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:43.747515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:43.747920 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:43.747922 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:43.747984 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:43.748590 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:43.748594 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:43.750766 (+ 2172us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":127,"threads_started":1}
W20260927 18:39:43.750938 21465 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:43.745491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:56489 (local address 127.11.251.254:41019)
0927 18:39:43.745620 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:43.745623 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:43.745715 (+ 92us) server_negotiation.cc:407] Connection header received
0927 18:39:43.745815 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:43.745816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:43.745832 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:43.745888 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:43.746269 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:43.746822 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:43.747609 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:43.747821 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:43.747912 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:43.750847 (+ 2935us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.194:56489: BlockingRecv error: recv got EOF from 127.11.251.194:56489 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 27 18:39:43 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:43 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:43.902647 21467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:43.897516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.195:55943)
0927 18:39:43.897681 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:43.897692 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:43.897776 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:43.898085 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:43.898086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:43.898363 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:43.898433 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:43.898436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:43.899276 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:43.899276 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:43.899837 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:43.899839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:43.900252 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:43.900253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:43.900297 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:43.900824 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:43.900829 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:43.902575 (+ 1746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":106,"threads_started":1}
W20260927 18:39:43.902837 21465 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:43.897627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:55943 (local address 127.11.251.254:41019)
0927 18:39:43.897704 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:43.897707 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:43.897753 (+ 46us) server_negotiation.cc:407] Connection header received
0927 18:39:43.897880 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:43.897882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:43.897902 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:43.897993 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:43.898539 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:43.899188 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:43.899957 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:43.900160 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:43.900238 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:43.902695 (+ 2457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.195:55943: BlockingRecv error: recv got EOF from 127.11.251.195:55943 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 27 18:39:44 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:44 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:44.611964 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:44.606665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.193:36965)
0927 18:39:44.606870 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:44.606887 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:44.606982 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:44.607244 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:44.607245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:44.607562 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:44.607628 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:44.607630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:44.608329 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:44.608330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:44.608950 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:44.608951 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:44.609316 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:44.609317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:44.609359 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:44.609879 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:44.609883 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:44.611871 (+ 1988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":100,"threads_started":1}
W20260927 18:39:44.612085 21469 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:44.606758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:36965 (local address 127.11.251.254:41019)
0927 18:39:44.607042 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:44.607045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:44.607056 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:39:44.607083 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:44.607084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:44.607104 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:44.607199 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:44.607712 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:44.608245 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:44.609036 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:44.609239 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:44.609298 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:44.611993 (+ 2695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.193:36965: BlockingRecv error: recv got EOF from 127.11.251.193:36965 (error 108)
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":116,"threads_started":1}
Sep 27 18:39:44 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:44 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:44.756819 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:44.751357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.194:53885)
0927 18:39:44.751580 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:44.751597 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:44.751709 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:44.751888 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:44.751889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:44.752120 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:44.752170 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:44.752173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:44.752896 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:44.752897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:44.753427 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:44.753428 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:44.753886 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:44.753887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:44.753943 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:44.754585 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:44.754590 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:44.756760 (+ 2170us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":112,"threads_started":1}
W20260927 18:39:44.757066 21469 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:44.751460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:53885 (local address 127.11.251.254:41019)
0927 18:39:44.751547 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:44.751549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:44.751685 (+ 136us) server_negotiation.cc:407] Connection header received
0927 18:39:44.751783 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:44.751784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:44.751800 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:44.751866 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:44.752254 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:44.752807 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:44.753512 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:44.753797 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:44.753885 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:44.756967 (+ 3082us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.194:53885: BlockingRecv error: recv got EOF from 127.11.251.194:53885 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 27 18:39:44 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:44 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:44.908658 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:44.903308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.195:52033)
0927 18:39:44.903546 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:44.903557 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:44.903637 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:44.903849 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:44.903850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:44.904067 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:44.904144 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:44.904146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:44.904816 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:44.904816 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:44.905398 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:44.905400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:44.905792 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:44.905793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:44.905859 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:44.906734 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:44.906741 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:44.908589 (+ 1848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":121,"threads_started":1}
W20260927 18:39:44.908762 21469 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:44.903410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:52033 (local address 127.11.251.254:41019)
0927 18:39:44.903504 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:44.903506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:44.903639 (+ 133us) server_negotiation.cc:407] Connection header received
0927 18:39:44.903683 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:44.903684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:44.903699 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:44.903773 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:44.904229 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:44.904732 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:44.905484 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:44.905703 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:44.905765 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:44.908702 (+ 2937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.195:52033: BlockingRecv error: recv got EOF from 127.11.251.195:52033 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 27 18:39:45 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:45 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:45.623459 21472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:45.612669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.193:53733)
0927 18:39:45.612863 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:45.612874 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:45.612929 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:45.613224 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:45.613225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:45.613405 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:45.613463 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:45.613464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:45.614469 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:45.614470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:45.615138 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:45.615140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:45.615542 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:45.615542 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:45.615587 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:45.620731 (+ 5144us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:45.620736 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:45.623320 (+ 2584us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":92,"threads_started":1}
W20260927 18:39:45.623514 21473 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:45.612795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:53733 (local address 127.11.251.254:41019)
0927 18:39:45.613007 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:45.613010 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:45.613025 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:39:45.613061 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:45.613062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:45.613077 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:45.613181 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:45.613592 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:45.614369 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:45.615225 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:45.615454 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:45.615530 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:45.623431 (+ 7901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.193:53733: BlockingRecv error: recv got EOF from 127.11.251.193:53733 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":125,"threads_started":1}
Sep 27 18:39:45 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:45 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:45.762784 21474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:45.757468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.194:56591)
0927 18:39:45.757677 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:45.757699 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:45.757789 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:45.757934 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:45.757935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:45.758147 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:45.758194 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:45.758195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:45.758889 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:45.758890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:45.759425 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:45.759426 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:45.759803 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:45.759804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:45.759865 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:45.760474 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:45.760480 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:45.762701 (+ 2221us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":106,"threads_started":1}
W20260927 18:39:45.762986 21473 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:45.757684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:56591 (local address 127.11.251.254:41019)
0927 18:39:45.757769 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:45.757771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:45.757782 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:39:45.757854 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:45.757856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:45.757870 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:45.757919 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:45.758278 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:45.758803 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:45.759511 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:45.759717 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:45.759794 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:45.762881 (+ 3087us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.194:56591: BlockingRecv error: recv got EOF from 127.11.251.194:56591 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 27 18:39:45 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:45 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:45.915200 21475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:45.909373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.195:55707)
0927 18:39:45.909662 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:45.909676 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:45.909782 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:45.910000 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:45.910000 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:45.910199 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:45.910249 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:45.910252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:45.911019 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:45.911020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:45.911967 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:45.911970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:45.912327 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:45.912329 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:45.912392 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:45.912887 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:45.912892 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:45.915117 (+ 2225us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":121,"threads_started":1}
W20260927 18:39:45.915326 21473 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:45.909497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:55707 (local address 127.11.251.254:41019)
0927 18:39:45.909640 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:45.909643 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:45.909757 (+ 114us) server_negotiation.cc:407] Connection header received
0927 18:39:45.909873 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:45.909875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:45.909897 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:45.909991 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:45.910343 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:45.910902 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:45.912053 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:45.912238 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:45.912301 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:45.915221 (+ 2920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.195:55707: BlockingRecv error: recv got EOF from 127.11.251.195:55707 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 27 18:39:46 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:46 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:46.629546 21476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:46.623970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.193:44027)
0927 18:39:46.624119 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:46.624131 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:46.624192 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:46.624501 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:46.624502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:46.624713 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:46.624768 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:46.624769 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:46.625613 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:46.625614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:46.626160 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:46.626162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:46.626665 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:46.626667 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:46.626775 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:46.627422 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:46.627435 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:46.629434 (+ 1999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
W20260927 18:39:46.629644 21477 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:46.624132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:44027 (local address 127.11.251.254:41019)
0927 18:39:46.624323 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:46.624325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:46.624336 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:39:46.624368 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:46.624369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:46.624385 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:46.624477 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:46.624859 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:46.625473 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:46.626274 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:46.626561 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:46.626647 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:46.629522 (+ 2875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.193:44027: BlockingRecv error: recv got EOF from 127.11.251.193:44027 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":101,"threads_started":1}
Sep 27 18:39:46 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:46 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:46.768656 21478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:46.763358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.194:33495)
0927 18:39:46.763607 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:46.763620 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:46.763721 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:46.763876 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:46.763877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:46.764086 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:46.764130 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:46.764131 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:46.764809 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:46.764810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:46.765352 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:46.765353 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:46.765791 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:46.765792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:46.765846 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:46.766355 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:46.766359 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:46.768578 (+ 2219us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":121,"threads_started":1}
W20260927 18:39:46.768913 21477 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:46.763474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:33495 (local address 127.11.251.254:41019)
0927 18:39:46.763567 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:46.763570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:46.763702 (+ 132us) server_negotiation.cc:407] Connection header received
0927 18:39:46.763783 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:46.763784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:46.763798 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:46.763870 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:46.764212 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:46.764728 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:46.765447 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:46.765705 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:46.765792 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:46.768783 (+ 2991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.194:33495: BlockingRecv error: recv got EOF from 127.11.251.194:33495 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 27 18:39:46 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:46 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.195@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:46.920766 21479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:46.915710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.195:42615)
0927 18:39:46.915922 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:46.915935 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:46.916030 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:46.916193 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:46.916194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:46.916385 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:46.916431 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:46.916433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:46.917152 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:46.917153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:46.917752 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:46.917753 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:46.918153 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:46.918154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:46.918222 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:46.918712 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:46.918717 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:46.920665 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":113,"threads_started":1}
W20260927 18:39:46.920825 21477 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:46.915817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:42615 (local address 127.11.251.254:41019)
0927 18:39:46.915907 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:46.915910 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:46.916032 (+ 122us) server_negotiation.cc:407] Connection header received
0927 18:39:46.916090 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:46.916091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:46.916109 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:46.916184 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:46.916515 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:46.917034 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:46.917831 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:46.918026 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:46.918137 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:46.920752 (+ 2615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.195:42615: BlockingRecv error: recv got EOF from 127.11.251.195:42615 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 27 18:39:47 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:47 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.193@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:47.635296 21480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:47.630197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.193:38133)
0927 18:39:47.630367 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:47.630386 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:47.630434 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:47.630651 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:47.630651 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:47.630823 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:47.630879 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:47.630880 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:47.631648 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:47.631648 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:47.632341 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:47.632343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:47.632735 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:47.632736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:47.632779 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:47.633414 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:47.633418 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:47.635222 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":115,"threads_started":1}
W20260927 18:39:47.635504 21481 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:47.630317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:38133 (local address 127.11.251.254:41019)
0927 18:39:47.630481 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:47.630483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:47.630494 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:39:47.630522 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:47.630523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:47.630536 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:47.630625 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:47.630985 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:47.631562 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:47.632430 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:47.632651 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:47.632728 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:47.635384 (+ 2656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.193:38133: BlockingRecv error: recv got EOF from 127.11.251.193:38133 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
Sep 27 18:39:47 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:39:47 dist-test-slave-z47j krb5kdc[20423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.251.194@KRBTEST.COM for oryx/127.11.251.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:39:47.775117 21482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:47.769429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41019 (local address 127.11.251.194:51105)
0927 18:39:47.769688 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:47.769706 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:47.769866 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:47.770061 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:47.770062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:47.770286 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:47.770348 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:47.770350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:47.771116 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:47.771117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:47.771858 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:47.771860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:47.772277 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:47.772278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:47.772331 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:47.772815 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:47.772820 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:47.775050 (+ 2230us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.251.254:41019: Server oryx/127.11.251.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":131,"threads_started":1}
W20260927 18:39:47.775300 21481 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:47.769632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:51105 (local address 127.11.251.254:41019)
0927 18:39:47.769701 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:47.769704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:47.769841 (+ 137us) server_negotiation.cc:407] Connection header received
0927 18:39:47.769949 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:47.769952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:47.769971 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:47.770037 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:47.770435 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:47.771010 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:47.771931 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:47.772186 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:47.772278 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:47.775189 (+ 2911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.251.194:51105: BlockingRecv error: recv got EOF from 127.11.251.194:51105 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 18:39:47.826231 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 21006
I20260927 18:39:47.832099 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 21159
I20260927 18:39:47.837205 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 21291
I20260927 18:39:47.842583 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 20936
2026-09-27T18:39:47Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17611 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-27T18:39:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:39:47Z Disabled control of system clock
I20260927 18:39:47.874265 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:36629
--webserver_interface=127.11.251.254
--webserver_port=0
--builtin_ntp_servers=127.11.251.212:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:36629
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260927 18:39:47.965852 21486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:47.966106 21486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:47.966128 21486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:47.967347 21486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:47.967386 21486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:47.967398 21486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:47.967411 21486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:47.967424 21486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:39:47.968780 21486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:36629
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:36629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:47.969089 21486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:47.969401 21486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:39:47.975379 21486 server_base.cc:1034] running on GCE node
W20260927 18:39:47.975354 21491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:47.975400 21494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:47.975334 21492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:47.976131 21486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:47.976480 21486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:47.977653 21486 hybrid_clock.cc:648] HybridClock initialized: now 1790534387977622 us; error 55 us; skew 500 ppm
I20260927 18:39:47.979177 21486 webserver.cc:460] Webserver started at http://127.11.251.254:43667/ using document root <none> and password file <none>
I20260927 18:39:47.979530 21486 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:47.979585 21486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:47.979750 21486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:47.980938 21486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "1f915b0f80d8416dbc52e73734def2f1"
format_stamp: "Formatted at 2026-09-27 18:39:47 on dist-test-slave-z47j"
server_key: "4b557761724df3e5af13325465f1bb4f"
server_key_iv: "c93b9e1c52a1f133614b1bf560f7c56e"
server_key_version: "encryptionkey@0"
I20260927 18:39:47.981334 21486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "1f915b0f80d8416dbc52e73734def2f1"
format_stamp: "Formatted at 2026-09-27 18:39:47 on dist-test-slave-z47j"
server_key: "4b557761724df3e5af13325465f1bb4f"
server_key_iv: "c93b9e1c52a1f133614b1bf560f7c56e"
server_key_version: "encryptionkey@0"
I20260927 18:39:47.982883 21486 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:39:47.983867 21500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:47.984139 21486 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:47.984259 21486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "1f915b0f80d8416dbc52e73734def2f1"
format_stamp: "Formatted at 2026-09-27 18:39:47 on dist-test-slave-z47j"
server_key: "4b557761724df3e5af13325465f1bb4f"
server_key_iv: "c93b9e1c52a1f133614b1bf560f7c56e"
server_key_version: "encryptionkey@0"
I20260927 18:39:47.984341 21486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:48.004729 21486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:48.005012 21486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:48.005127 21486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:48.009388 21486 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:36629
I20260927 18:39:48.009510 21552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:36629 every 8 connection(s)
I20260927 18:39:48.009887 21486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:39:48.010776 21553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:48.013918 21553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1: Bootstrap starting.
I20260927 18:39:48.014631 21553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:48.015000 21553 log.cc:824] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:48.015970 21553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1: No bootstrap required, opened a new log
I20260927 18:39:48.017987 21553 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f915b0f80d8416dbc52e73734def2f1" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 36629 } }
I20260927 18:39:48.018186 21553 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:48.018235 21553 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f915b0f80d8416dbc52e73734def2f1, State: Initialized, Role: FOLLOWER
I20260927 18:39:48.018448 21553 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f915b0f80d8416dbc52e73734def2f1" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 36629 } }
I20260927 18:39:48.018554 21553 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:39:48.018602 21553 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:39:48.018656 21553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:48.019340 21553 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f915b0f80d8416dbc52e73734def2f1" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 36629 } }
I20260927 18:39:48.019472 21553 leader_election.cc:302] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f915b0f80d8416dbc52e73734def2f1; no voters:
I20260927 18:39:48.019752 21553 leader_election.cc:288] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:39:48.019938 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 21486
I20260927 18:39:48.019954 21556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:48.020043 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:39:48.020113 21553 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:48.020287 12271 external_mini_cluster.cc:1428] Setting key 617f5d4b5867d9cf8539187e4fdb9165
I20260927 18:39:48.020251 21556 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1 [term 1 LEADER]: Becoming Leader. State: Replica: 1f915b0f80d8416dbc52e73734def2f1, State: Running, Role: LEADER
I20260927 18:39:48.020406 21556 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f915b0f80d8416dbc52e73734def2f1" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 36629 } }
I20260927 18:39:48.021147 21557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f915b0f80d8416dbc52e73734def2f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f915b0f80d8416dbc52e73734def2f1" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 36629 } } }
I20260927 18:39:48.021217 21557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:48.021101 21558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f915b0f80d8416dbc52e73734def2f1. Latest consensus state: current_term: 1 leader_uuid: "1f915b0f80d8416dbc52e73734def2f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f915b0f80d8416dbc52e73734def2f1" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 36629 } } }
I20260927 18:39:48.021473 21558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:48.021471 21567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:39:48.022261 21567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:39:48.024276 21567 catalog_manager.cc:1347] Generated new cluster ID: 7bc13e0bc5324600b4bdd76ad0ba634e
I20260927 18:39:48.024350 21567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:39:48.027041 21563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:48.020913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46872 (local address 127.11.251.254:36629)
0927 18:39:48.022230 (+ 1317us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:48.022242 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:48.022281 (+ 39us) server_negotiation.cc:407] Connection header received
0927 18:39:48.022542 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:48.022553 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:48.022636 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:48.022825 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:48.023065 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:48.024145 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:48.024998 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:48.025468 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:48.025618 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:48.026356 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:48.026360 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:48.026360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:48.026547 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:48.026581 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:48.026592 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:48.026682 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:48.026742 (+ 60us) server_negotiation.cc:299] Negotiation successful
0927 18:39:48.026827 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1218,"thread_start_us":101,"threads_started":1}
I20260927 18:39:48.032452 21567 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:39:48.033011 21567 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:39:48.039143 21567 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f915b0f80d8416dbc52e73734def2f1: Generated new TSK 0
I20260927 18:39:48.039323 21567 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:39:48.041319 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:36629
--builtin_ntp_servers=127.11.251.212:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260927 18:39:48.127591 21577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:48.127789 21577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:48.127808 21577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:48.128957 21577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:48.128998 21577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:48.129024 21577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:39:48.130424 21577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:36629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:48.130685 21577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:48.131003 21577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:48.136214 21582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:48.136461 21577 server_base.cc:1034] running on GCE node
W20260927 18:39:48.136339 21583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:48.136471 21585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:48.136840 21577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:48.137096 21577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:48.138257 21577 hybrid_clock.cc:648] HybridClock initialized: now 1790534388138225 us; error 54 us; skew 500 ppm
I20260927 18:39:48.139521 21577 webserver.cc:460] Webserver started at http://127.11.251.193:45719/ using document root <none> and password file <none>
I20260927 18:39:48.139768 21577 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:48.139838 21577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:48.139974 21577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:48.141023 21577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "597262652c534f5b86cc45c61c66f345"
format_stamp: "Formatted at 2026-09-27 18:39:48 on dist-test-slave-z47j"
server_key: "0df6f7d49ac1f1ce65e48ddfa53a65d9"
server_key_iv: "bc97454660d19365a9c350807291c8ee"
server_key_version: "encryptionkey@0"
I20260927 18:39:48.141387 21577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "597262652c534f5b86cc45c61c66f345"
format_stamp: "Formatted at 2026-09-27 18:39:48 on dist-test-slave-z47j"
server_key: "0df6f7d49ac1f1ce65e48ddfa53a65d9"
server_key_iv: "bc97454660d19365a9c350807291c8ee"
server_key_version: "encryptionkey@0"
I20260927 18:39:48.142838 21577 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:39:48.143679 21591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:48.143901 21577 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:39:48.143996 21577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "597262652c534f5b86cc45c61c66f345"
format_stamp: "Formatted at 2026-09-27 18:39:48 on dist-test-slave-z47j"
server_key: "0df6f7d49ac1f1ce65e48ddfa53a65d9"
server_key_iv: "bc97454660d19365a9c350807291c8ee"
server_key_version: "encryptionkey@0"
I20260927 18:39:48.144084 21577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:48.169641 21577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:48.169976 21577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:48.170132 21577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:48.170369 21577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:48.170749 21577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:48.170806 21577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:48.170867 21577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:48.170917 21577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:48.178045 21577 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:43729
I20260927 18:39:48.178098 21704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:43729 every 8 connection(s)
I20260927 18:39:48.178409 21577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:39:48.183866 21563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:48.179257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:60867 (local address 127.11.251.254:36629)
0927 18:39:48.179361 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:48.179365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:48.179696 (+ 331us) server_negotiation.cc:407] Connection header received
0927 18:39:48.180239 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:48.180241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:48.180256 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:48.180355 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:48.180992 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:48.181545 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:48.182548 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:48.182738 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:48.182798 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:48.183620 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:48.183621 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:48.183622 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:48.183627 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:48.183640 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:48.183644 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:48.183686 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:48.183762 (+ 76us) server_negotiation.cc:299] Negotiation successful
0927 18:39:48.183790 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 18:39:48.184020 21707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:48.179248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:36629 (local address 127.11.251.193:60867)
0927 18:39:48.179540 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:48.179581 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:48.180112 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:48.180412 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:48.180415 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:48.180423 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:48.180899 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:48.180903 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:48.181687 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:48.181688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:48.182463 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:48.182465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:48.182822 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:48.182823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:48.182929 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:48.183494 (+ 565us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:39:48.183494 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:48.183520 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:39:48.183541 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:48.183694 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:48.183697 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:48.183699 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:39:48.183748 (+ 49us) client_negotiation.cc:240] Negotiation successful
0927 18:39:48.183819 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":169,"threads_started":1}
I20260927 18:39:48.184615 21705 heartbeater.cc:347] Connected to a master server at 127.11.251.254:36629
I20260927 18:39:48.184703 21705 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:48.184839 21705 heartbeater.cc:511] Master 127.11.251.254:36629 requested a full tablet report, sending...
I20260927 18:39:48.185415 21517 ts_manager.cc:194] Registered new tserver with Master: 597262652c534f5b86cc45c61c66f345 (127.11.251.193:43729)
I20260927 18:39:48.186089 21517 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.251.193:60867
I20260927 18:39:48.187063 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 21577
I20260927 18:39:48.187146 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:39:48.187358 12271 external_mini_cluster.cc:1428] Setting key 27dcddfeb0ebdbe44fcea7f58f104ff3
I20260927 18:39:48.188386 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:36629
--builtin_ntp_servers=127.11.251.212:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260927 18:39:48.285029 21708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:48.285262 21708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:48.285284 21708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:48.286595 21708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:48.286636 21708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:48.286661 21708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:39:48.288162 21708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:36629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:48.288432 21708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:48.288787 21708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:48.294404 21716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:48.294428 21713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:48.294452 21714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:48.294761 21708 server_base.cc:1034] running on GCE node
I20260927 18:39:48.295048 21708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:48.295271 21708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:48.296435 21708 hybrid_clock.cc:648] HybridClock initialized: now 1790534388296423 us; error 30 us; skew 500 ppm
I20260927 18:39:48.297981 21708 webserver.cc:460] Webserver started at http://127.11.251.194:46777/ using document root <none> and password file <none>
I20260927 18:39:48.298208 21708 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:48.298257 21708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:48.298360 21708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:48.299479 21708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "57077cf672e54ba9b465ba2c73ae581f"
format_stamp: "Formatted at 2026-09-27 18:39:48 on dist-test-slave-z47j"
server_key: "217870e3cb4dc8433e19a56a8b2a6c39"
server_key_iv: "5cb0fdd7b4dd7f6b4ca64adb7a67bb45"
server_key_version: "encryptionkey@0"
I20260927 18:39:48.299808 21708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "57077cf672e54ba9b465ba2c73ae581f"
format_stamp: "Formatted at 2026-09-27 18:39:48 on dist-test-slave-z47j"
server_key: "217870e3cb4dc8433e19a56a8b2a6c39"
server_key_iv: "5cb0fdd7b4dd7f6b4ca64adb7a67bb45"
server_key_version: "encryptionkey@0"
I20260927 18:39:48.301111 21708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:39:48.301903 21722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:48.302094 21708 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:48.302203 21708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "57077cf672e54ba9b465ba2c73ae581f"
format_stamp: "Formatted at 2026-09-27 18:39:48 on dist-test-slave-z47j"
server_key: "217870e3cb4dc8433e19a56a8b2a6c39"
server_key_iv: "5cb0fdd7b4dd7f6b4ca64adb7a67bb45"
server_key_version: "encryptionkey@0"
I20260927 18:39:48.302284 21708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:48.332580 21708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:48.332967 21708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:48.333119 21708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:48.333402 21708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:48.333827 21708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:48.333884 21708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:48.333931 21708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:48.333962 21708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:48.340737 21708 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:42201
I20260927 18:39:48.340821 21835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:42201 every 8 connection(s)
I20260927 18:39:48.341231 21708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:39:48.344246 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 21708
I20260927 18:39:48.344339 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:39:48.344566 12271 external_mini_cluster.cc:1428] Setting key 0b525ac9e167e26914338f40a1004613
I20260927 18:39:48.345734 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:36629
--builtin_ntp_servers=127.11.251.212:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260927 18:39:48.347961 21563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:48.342178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:55651 (local address 127.11.251.254:36629)
0927 18:39:48.342259 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:48.342262 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:48.342569 (+ 307us) server_negotiation.cc:407] Connection header received
0927 18:39:48.343167 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:48.343168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:48.343180 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:48.343251 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:48.344011 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:48.344661 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:48.346156 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:48.346395 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:48.346464 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:48.347557 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:48.347559 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:48.347559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:48.347566 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:48.347586 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:48.347592 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:48.347662 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:48.347795 (+ 133us) server_negotiation.cc:299] Negotiation successful
0927 18:39:48.347829 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:39:48.348098 21838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:48.342162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:36629 (local address 127.11.251.194:55651)
0927 18:39:48.342395 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:48.342442 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:48.343058 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:48.343330 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:48.343333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:48.343343 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:48.343923 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:48.343927 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:48.344739 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:48.344741 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:48.346060 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:48.346063 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:48.346494 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:48.346495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:48.346665 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:48.347361 (+ 696us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:39:48.347362 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:48.347407 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:39:48.347448 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:48.347673 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:48.347678 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:48.347681 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:39:48.347755 (+ 74us) client_negotiation.cc:240] Negotiation successful
0927 18:39:48.347856 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":115,"threads_started":1}
I20260927 18:39:48.348647 21836 heartbeater.cc:347] Connected to a master server at 127.11.251.254:36629
I20260927 18:39:48.348775 21836 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:48.349006 21836 heartbeater.cc:511] Master 127.11.251.254:36629 requested a full tablet report, sending...
I20260927 18:39:48.349437 21517 ts_manager.cc:194] Registered new tserver with Master: 57077cf672e54ba9b465ba2c73ae581f (127.11.251.194:42201)
I20260927 18:39:48.350013 21517 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.251.194:55651
W20260927 18:39:48.438205 21839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:48.438473 21839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:48.438496 21839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:48.439694 21839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:48.439733 21839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:48.439759 21839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:39:48.441071 21839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:36629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:48.441295 21839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:48.441632 21839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:48.447520 21844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:48.447521 21845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:48.447489 21847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:48.447922 21839 server_base.cc:1034] running on GCE node
I20260927 18:39:48.448127 21839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:48.448355 21839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:48.449504 21839 hybrid_clock.cc:648] HybridClock initialized: now 1790534388449493 us; error 32 us; skew 500 ppm
I20260927 18:39:48.450959 21839 webserver.cc:460] Webserver started at http://127.11.251.195:43997/ using document root <none> and password file <none>
I20260927 18:39:48.451175 21839 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:48.451218 21839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:48.451344 21839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:48.452622 21839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "fd2f5e9d25b24b33b3ad8946322a4238"
format_stamp: "Formatted at 2026-09-27 18:39:48 on dist-test-slave-z47j"
server_key: "ca2e65790f11256feb97f30dfbd43f12"
server_key_iv: "72621bff2748441a5659db733e5f8910"
server_key_version: "encryptionkey@0"
I20260927 18:39:48.452960 21839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "fd2f5e9d25b24b33b3ad8946322a4238"
format_stamp: "Formatted at 2026-09-27 18:39:48 on dist-test-slave-z47j"
server_key: "ca2e65790f11256feb97f30dfbd43f12"
server_key_iv: "72621bff2748441a5659db733e5f8910"
server_key_version: "encryptionkey@0"
I20260927 18:39:48.454360 21839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:39:48.455106 21853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:48.455327 21839 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:48.455407 21839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "fd2f5e9d25b24b33b3ad8946322a4238"
format_stamp: "Formatted at 2026-09-27 18:39:48 on dist-test-slave-z47j"
server_key: "ca2e65790f11256feb97f30dfbd43f12"
server_key_iv: "72621bff2748441a5659db733e5f8910"
server_key_version: "encryptionkey@0"
I20260927 18:39:48.455466 21839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:48.491649 21839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:48.492014 21839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:48.492130 21839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:48.492359 21839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:48.492753 21839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:48.492822 21839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:48.492872 21839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:48.492893 21839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:48.499646 21839 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:37805
I20260927 18:39:48.499696 21966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:37805 every 8 connection(s)
I20260927 18:39:48.500092 21839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:39:48.503093 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 21839
I20260927 18:39:48.503171 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:39:48.503405 12271 external_mini_cluster.cc:1428] Setting key e0044f53253b0f45c1bdd927d1fe1538
I20260927 18:39:48.506492 21563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:48.500956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:50819 (local address 127.11.251.254:36629)
0927 18:39:48.501041 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:48.501043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:48.501281 (+ 238us) server_negotiation.cc:407] Connection header received
0927 18:39:48.502031 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:48.502033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:48.502044 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:48.502101 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:48.502817 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:48.503512 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:48.504864 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:48.505074 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:48.505150 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:48.506185 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:48.506186 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:48.506187 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:48.506192 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:48.506213 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:48.506219 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:48.506290 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:48.506389 (+ 99us) server_negotiation.cc:299] Negotiation successful
0927 18:39:48.506423 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 18:39:48.506744 21969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:48.500884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:36629 (local address 127.11.251.195:50819)
0927 18:39:48.501144 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:48.501192 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:48.501929 (+ 737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:48.502181 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:48.502184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:48.502194 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:48.502736 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:48.502740 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:48.503717 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:48.503718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:48.504727 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:48.504729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:48.505191 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:48.505193 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:48.505325 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:48.506039 (+ 714us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:39:48.506039 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:48.506072 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:39:48.506097 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:48.506304 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:48.506309 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:48.506311 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:39:48.506382 (+ 71us) client_negotiation.cc:240] Negotiation successful
0927 18:39:48.506474 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":92,"threads_started":1}
I20260927 18:39:48.507292 21967 heartbeater.cc:347] Connected to a master server at 127.11.251.254:36629
I20260927 18:39:48.507426 21967 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:48.507654 21967 heartbeater.cc:511] Master 127.11.251.254:36629 requested a full tablet report, sending...
I20260927 18:39:48.508229 21517 ts_manager.cc:194] Registered new tserver with Master: fd2f5e9d25b24b33b3ad8946322a4238 (127.11.251.195:37805)
I20260927 18:39:48.508724 21517 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.251.195:50819
I20260927 18:39:48.515419 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 18:39:48.517730 21563 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:48.517089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46874 (local address 127.11.251.254:36629)
0927 18:39:48.517162 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:48.517165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:48.517280 (+ 115us) server_negotiation.cc:407] Connection header received
0927 18:39:48.517365 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:48.517366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:48.517383 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:48.517442 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:48.517642 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46874: BlockingRecv error: recv got EOF from 127.0.0.1:46874 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 18:39:48.517786 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 21577
I20260927 18:39:48.523663 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 21708
I20260927 18:39:48.528875 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 21839
I20260927 18:39:48.533777 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 21486
2026-09-27T18:39:48Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (688 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-27T18:39:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:39:48Z Disabled control of system clock
I20260927 18:39:48.562716 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:33003
--webserver_interface=127.11.251.254
--webserver_port=0
--builtin_ntp_servers=127.11.251.212:37539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:33003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 18:39:48.656250 21978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:48.656504 21978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:48.656527 21978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:48.657817 21978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:48.657858 21978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:48.657871 21978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:48.657889 21978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:48.657902 21978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:39:48.659231 21978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:37539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:33003
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:33003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:48.659464 21978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:48.659754 21978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:48.664985 21984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:48.665014 21986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:48.665304 21978 server_base.cc:1034] running on GCE node
W20260927 18:39:48.665038 21983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:48.665678 21978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:48.665964 21978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:48.667119 21978 hybrid_clock.cc:648] HybridClock initialized: now 1790534388667096 us; error 40 us; skew 500 ppm
I20260927 18:39:48.668524 21978 webserver.cc:460] Webserver started at http://127.11.251.254:34683/ using document root <none> and password file <none>
I20260927 18:39:48.668787 21978 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:48.668859 21978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:48.669039 21978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:48.670276 21978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "9c4b7e5d083849f4a65964e5e735e585"
format_stamp: "Formatted at 2026-09-27 18:39:48 on dist-test-slave-z47j"
server_key: "70a414d033d12d18a6e33e9643bd4328"
server_key_iv: "728e45f4d498034a7c57d47135229617"
server_key_version: "encryptionkey@0"
I20260927 18:39:48.670696 21978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "9c4b7e5d083849f4a65964e5e735e585"
format_stamp: "Formatted at 2026-09-27 18:39:48 on dist-test-slave-z47j"
server_key: "70a414d033d12d18a6e33e9643bd4328"
server_key_iv: "728e45f4d498034a7c57d47135229617"
server_key_version: "encryptionkey@0"
I20260927 18:39:48.672117 21978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260927 18:39:48.672969 21992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:48.673182 21978 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:48.673300 21978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "9c4b7e5d083849f4a65964e5e735e585"
format_stamp: "Formatted at 2026-09-27 18:39:48 on dist-test-slave-z47j"
server_key: "70a414d033d12d18a6e33e9643bd4328"
server_key_iv: "728e45f4d498034a7c57d47135229617"
server_key_version: "encryptionkey@0"
I20260927 18:39:48.673398 21978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:48.677126 21978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:48.677459 21978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:48.677656 21978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:48.682942 21978 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:33003
I20260927 18:39:48.682973 22044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:33003 every 8 connection(s)
I20260927 18:39:48.683355 21978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:39:48.684111 22045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:48.687356 22045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585: Bootstrap starting.
I20260927 18:39:48.687678 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 21978
I20260927 18:39:48.687808 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:39:48.688015 12271 external_mini_cluster.cc:1428] Setting key 5a8e3efa19fb07328cc914bc69976902
I20260927 18:39:48.688179 22045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:48.688537 22045 log.cc:824] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:48.689522 22045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585: No bootstrap required, opened a new log
I20260927 18:39:48.691284 22048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:48.688635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48372 (local address 127.11.251.254:33003)
0927 18:39:48.688875 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:48.688886 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:48.688923 (+ 37us) server_negotiation.cc:407] Connection header received
0927 18:39:48.689514 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:48.689523 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:48.689566 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:48.689794 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:48.690803 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:48.690807 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:48.690807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:48.690812 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:48.690866 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:48.690882 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:48.691002 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:48.691039 (+ 37us) server_negotiation.cc:299] Negotiation successful
0927 18:39:48.691102 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":98,"threads_started":1}
I20260927 18:39:48.691529 22045 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4b7e5d083849f4a65964e5e735e585" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 33003 } }
I20260927 18:39:48.691717 22045 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:48.691766 22045 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c4b7e5d083849f4a65964e5e735e585, State: Initialized, Role: FOLLOWER
I20260927 18:39:48.691928 22045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4b7e5d083849f4a65964e5e735e585" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 33003 } }
I20260927 18:39:48.692014 22045 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:39:48.692046 22045 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:39:48.692104 22045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:48.693060 22045 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4b7e5d083849f4a65964e5e735e585" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 33003 } }
I20260927 18:39:48.693190 22045 leader_election.cc:302] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c4b7e5d083849f4a65964e5e735e585; no voters:
I20260927 18:39:48.693392 22045 leader_election.cc:288] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:39:48.693486 22050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:48.693723 22045 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:48.693795 22050 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585 [term 1 LEADER]: Becoming Leader. State: Replica: 9c4b7e5d083849f4a65964e5e735e585, State: Running, Role: LEADER
I20260927 18:39:48.694003 22050 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4b7e5d083849f4a65964e5e735e585" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 33003 } }
I20260927 18:39:48.694595 22052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c4b7e5d083849f4a65964e5e735e585. Latest consensus state: current_term: 1 leader_uuid: "9c4b7e5d083849f4a65964e5e735e585" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4b7e5d083849f4a65964e5e735e585" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 33003 } } }
I20260927 18:39:48.694694 22052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:48.694793 22051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c4b7e5d083849f4a65964e5e735e585" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4b7e5d083849f4a65964e5e735e585" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 33003 } } }
I20260927 18:39:48.694851 22051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:48.694971 22059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:39:48.695832 22059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:39:48.697463 22059 catalog_manager.cc:1347] Generated new cluster ID: 286c81dfdea34888bd32d43e13a7b92b
I20260927 18:39:48.697515 22059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:39:48.708642 22059 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:39:48.708856 22059 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:39:48.712224 22059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c4b7e5d083849f4a65964e5e735e585: Generated new TSK 0
I20260927 18:39:48.712467 22059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:39:48.713984 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:33003
--builtin_ntp_servers=127.11.251.212:37539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 18:39:48.811738 22069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:48.812023 22069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:48.812062 22069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:48.813381 22069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:48.813449 22069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:48.813499 22069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:39:48.814976 22069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:37539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:33003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:48.815269 22069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:48.815613 22069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:48.821192 22074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:48.821223 22075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:48.821295 22077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:48.821342 22069 server_base.cc:1034] running on GCE node
I20260927 18:39:48.821671 22069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:48.821949 22069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:48.823123 22069 hybrid_clock.cc:648] HybridClock initialized: now 1790534388823101 us; error 40 us; skew 500 ppm
I20260927 18:39:48.824560 22069 webserver.cc:460] Webserver started at http://127.11.251.193:38745/ using document root <none> and password file <none>
I20260927 18:39:48.824843 22069 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:48.824899 22069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:48.825073 22069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:48.826321 22069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "e6d79a801e244ea3b89bb1f4a1aefe52"
format_stamp: "Formatted at 2026-09-27 18:39:48 on dist-test-slave-z47j"
server_key: "d99bbbc67c84df681a8a9f54437af5f2"
server_key_iv: "335bd5f31c5bf1e2f73ff1c3e0375eb5"
server_key_version: "encryptionkey@0"
I20260927 18:39:48.826710 22069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "e6d79a801e244ea3b89bb1f4a1aefe52"
format_stamp: "Formatted at 2026-09-27 18:39:48 on dist-test-slave-z47j"
server_key: "d99bbbc67c84df681a8a9f54437af5f2"
server_key_iv: "335bd5f31c5bf1e2f73ff1c3e0375eb5"
server_key_version: "encryptionkey@0"
I20260927 18:39:48.828168 22069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:39:48.829051 22083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:48.829232 22069 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:48.829335 22069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "e6d79a801e244ea3b89bb1f4a1aefe52"
format_stamp: "Formatted at 2026-09-27 18:39:48 on dist-test-slave-z47j"
server_key: "d99bbbc67c84df681a8a9f54437af5f2"
server_key_iv: "335bd5f31c5bf1e2f73ff1c3e0375eb5"
server_key_version: "encryptionkey@0"
I20260927 18:39:48.829435 22069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:48.832918 22069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:48.833212 22069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:48.833356 22069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:48.833632 22069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:48.833963 22069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:48.834017 22069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:48.834057 22069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:48.834084 22069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:48.840458 22069 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:42061
I20260927 18:39:48.840508 22196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:42061 every 8 connection(s)
I20260927 18:39:48.840919 22069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:39:48.844194 22048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:48.841762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:48153 (local address 127.11.251.254:33003)
0927 18:39:48.841855 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:48.841857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:48.842145 (+ 288us) server_negotiation.cc:407] Connection header received
0927 18:39:48.842637 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:48.842638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:48.842645 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:48.842704 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:48.843923 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:48.843925 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:48.843925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:48.843929 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:48.843951 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:48.843956 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:48.844018 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:48.844100 (+ 82us) server_negotiation.cc:299] Negotiation successful
0927 18:39:48.844114 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 18:39:48.844372 22199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:48.841798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:33003 (local address 127.11.251.193:48153)
0927 18:39:48.841976 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:48.842011 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:48.842551 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:48.842763 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:48.842766 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:48.842775 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:48.843724 (+ 949us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:39:48.843724 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:48.843754 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:39:48.843786 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:48.844027 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:48.844032 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:48.844034 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:39:48.844094 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 18:39:48.844146 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260927 18:39:48.845062 22197 heartbeater.cc:347] Connected to a master server at 127.11.251.254:33003
I20260927 18:39:48.845170 22197 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:48.845336 22197 heartbeater.cc:511] Master 127.11.251.254:33003 requested a full tablet report, sending...
I20260927 18:39:48.845949 22009 ts_manager.cc:194] Registered new tserver with Master: e6d79a801e244ea3b89bb1f4a1aefe52 (127.11.251.193:42061)
I20260927 18:39:48.850656 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 22069
I20260927 18:39:48.850744 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:39:48.850932 12271 external_mini_cluster.cc:1428] Setting key f3b191ec56aef54230a0b57e6950dfd8
I20260927 18:39:48.852010 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:33003
--builtin_ntp_servers=127.11.251.212:37539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 18:39:48.942667 22200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:48.942924 22200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:48.942957 22200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:48.944231 22200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:48.944273 22200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:48.944301 22200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:39:48.945724 22200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:37539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:33003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:48.945977 22200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:48.946331 22200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:48.952034 22205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:48.952034 22206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:48.952061 22208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:48.952322 22200 server_base.cc:1034] running on GCE node
I20260927 18:39:48.952569 22200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:48.952888 22200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:48.954058 22200 hybrid_clock.cc:648] HybridClock initialized: now 1790534388954037 us; error 45 us; skew 500 ppm
I20260927 18:39:48.955579 22200 webserver.cc:460] Webserver started at http://127.11.251.194:35085/ using document root <none> and password file <none>
I20260927 18:39:48.955891 22200 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:48.955983 22200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:48.956151 22200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:48.957849 22200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "691906066d65492e8acde21abae1ab67"
format_stamp: "Formatted at 2026-09-27 18:39:48 on dist-test-slave-z47j"
server_key: "2f4beb846d9ca0f663159ea5e257a0a2"
server_key_iv: "7c4d564b900ba0971cc2b228b90e326d"
server_key_version: "encryptionkey@0"
I20260927 18:39:48.958261 22200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "691906066d65492e8acde21abae1ab67"
format_stamp: "Formatted at 2026-09-27 18:39:48 on dist-test-slave-z47j"
server_key: "2f4beb846d9ca0f663159ea5e257a0a2"
server_key_iv: "7c4d564b900ba0971cc2b228b90e326d"
server_key_version: "encryptionkey@0"
I20260927 18:39:48.959676 22200 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:39:48.960460 22214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:48.960644 22200 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:39:48.960760 22200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "691906066d65492e8acde21abae1ab67"
format_stamp: "Formatted at 2026-09-27 18:39:48 on dist-test-slave-z47j"
server_key: "2f4beb846d9ca0f663159ea5e257a0a2"
server_key_iv: "7c4d564b900ba0971cc2b228b90e326d"
server_key_version: "encryptionkey@0"
I20260927 18:39:48.960843 22200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:48.964534 22200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:48.964831 22200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:48.964980 22200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:48.965204 22200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:48.965556 22200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:48.965641 22200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:48.965689 22200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:48.965716 22200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:48.972652 22200 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:39981
I20260927 18:39:48.972697 22327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:39981 every 8 connection(s)
I20260927 18:39:48.973099 22200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:39:48.976554 22048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:48.974002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:47689 (local address 127.11.251.254:33003)
0927 18:39:48.974084 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:48.974087 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:48.974315 (+ 228us) server_negotiation.cc:407] Connection header received
0927 18:39:48.974908 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:48.974909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:48.974919 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:48.974971 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:48.976217 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:48.976219 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:48.976219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:48.976224 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:48.976244 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:48.976249 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:48.976336 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:48.976450 (+ 114us) server_negotiation.cc:299] Negotiation successful
0927 18:39:48.976469 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:39:48.976769 22330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:48.973996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:33003 (local address 127.11.251.194:47689)
0927 18:39:48.974183 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:48.974221 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:48.974794 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:48.975054 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:48.975058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:48.975067 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:48.976041 (+ 974us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:39:48.976041 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:48.976073 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:39:48.976119 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:48.976332 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:48.976338 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:48.976341 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:39:48.976449 (+ 108us) client_negotiation.cc:240] Negotiation successful
0927 18:39:48.976519 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260927 18:39:48.977334 22328 heartbeater.cc:347] Connected to a master server at 127.11.251.254:33003
I20260927 18:39:48.977418 22328 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:48.977561 22328 heartbeater.cc:511] Master 127.11.251.254:33003 requested a full tablet report, sending...
I20260927 18:39:48.977775 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 22200
I20260927 18:39:48.977946 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:39:48.978083 22009 ts_manager.cc:194] Registered new tserver with Master: 691906066d65492e8acde21abae1ab67 (127.11.251.194:39981)
I20260927 18:39:48.978174 12271 external_mini_cluster.cc:1428] Setting key 0561c1ae47b68adc493fb48fc87d8a88
I20260927 18:39:48.979193 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:33003
--builtin_ntp_servers=127.11.251.212:37539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 18:39:49.068575 22331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:49.068853 22331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:49.068902 22331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:49.070341 22331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:49.070411 22331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:49.070467 22331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:39:49.071913 22331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:37539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:33003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:49.072211 22331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:49.072537 22331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:49.078248 22336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:49.078477 22331 server_base.cc:1034] running on GCE node
W20260927 18:39:49.078249 22337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:49.078249 22339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:49.078843 22331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:49.079146 22331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:49.080317 22331 hybrid_clock.cc:648] HybridClock initialized: now 1790534389080315 us; error 44 us; skew 500 ppm
I20260927 18:39:49.081967 22331 webserver.cc:460] Webserver started at http://127.11.251.195:45305/ using document root <none> and password file <none>
I20260927 18:39:49.082284 22331 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:49.082357 22331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:49.082527 22331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:49.083806 22331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "08c55d87ea5746ba90d1a8a16b7d7b88"
format_stamp: "Formatted at 2026-09-27 18:39:49 on dist-test-slave-z47j"
server_key: "1cd5ea41dc7d08aa71b608e6275b5aca"
server_key_iv: "0e5a6a658bb15dda9d3fa06b0760cf8a"
server_key_version: "encryptionkey@0"
I20260927 18:39:49.084266 22331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "08c55d87ea5746ba90d1a8a16b7d7b88"
format_stamp: "Formatted at 2026-09-27 18:39:49 on dist-test-slave-z47j"
server_key: "1cd5ea41dc7d08aa71b608e6275b5aca"
server_key_iv: "0e5a6a658bb15dda9d3fa06b0760cf8a"
server_key_version: "encryptionkey@0"
I20260927 18:39:49.085819 22331 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260927 18:39:49.086704 22345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:49.086944 22331 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:49.087042 22331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "08c55d87ea5746ba90d1a8a16b7d7b88"
format_stamp: "Formatted at 2026-09-27 18:39:49 on dist-test-slave-z47j"
server_key: "1cd5ea41dc7d08aa71b608e6275b5aca"
server_key_iv: "0e5a6a658bb15dda9d3fa06b0760cf8a"
server_key_version: "encryptionkey@0"
I20260927 18:39:49.087124 22331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:49.090935 22331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:49.091264 22331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:49.091439 22331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:49.091660 22331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:49.092020 22331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:49.092060 22331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:49.092084 22331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:49.092096 22331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:49.099929 22331 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:37159
I20260927 18:39:49.099987 22458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:37159 every 8 connection(s)
I20260927 18:39:49.100530 22331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:39:49.104931 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 22331
I20260927 18:39:49.105202 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:39:49.105538 12271 external_mini_cluster.cc:1428] Setting key 36ffc06bf65722805b9c22cc0d7170e0
I20260927 18:39:49.106240 22048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:49.102228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:44971 (local address 127.11.251.254:33003)
0927 18:39:49.102293 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:49.102296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:49.102665 (+ 369us) server_negotiation.cc:407] Connection header received
0927 18:39:49.103245 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:49.103247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:49.103256 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:49.103355 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:49.105425 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:49.105428 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:49.105429 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:39:49.105438 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:49.105462 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:39:49.105469 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:49.105559 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:49.106141 (+ 582us) server_negotiation.cc:299] Negotiation successful
0927 18:39:49.106162 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 18:39:49.106402 22461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:49.102109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:33003 (local address 127.11.251.195:44971)
0927 18:39:49.102370 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:49.102429 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:49.103032 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:49.103523 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:49.103529 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:49.103540 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:49.105068 (+ 1528us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:39:49.105069 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:49.105138 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:39:49.105180 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:49.105879 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:49.105888 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:49.105892 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:39:49.106008 (+ 116us) client_negotiation.cc:240] Negotiation successful
0927 18:39:49.106081 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":124,"threads_started":1}
I20260927 18:39:49.107199 22459 heartbeater.cc:347] Connected to a master server at 127.11.251.254:33003
I20260927 18:39:49.107306 22459 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:49.107594 22459 heartbeater.cc:511] Master 127.11.251.254:33003 requested a full tablet report, sending...
I20260927 18:39:49.108197 22008 ts_manager.cc:194] Registered new tserver with Master: 08c55d87ea5746ba90d1a8a16b7d7b88 (127.11.251.195:37159)
I20260927 18:39:49.118860 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 18:39:49.121541 22048 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:39:49.120892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48376 (local address 127.11.251.254:33003)
0927 18:39:49.120944 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:49.120946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:49.121033 (+ 87us) server_negotiation.cc:407] Connection header received
0927 18:39:49.121085 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:49.121085 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:49.121092 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:49.121143 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:49.121428 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48376: BlockingRecv error: recv got EOF from 127.0.0.1:48376 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 18:39:49.121656 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 22069
I20260927 18:39:49.127621 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 22200
I20260927 18:39:49.132427 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 22331
I20260927 18:39:49.137328 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 21978
2026-09-27T18:39:49Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (603 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:39:49 dist-test-slave-z47j krb5kdc[22469](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:39:49 dist-test-slave-z47j krb5kdc[22469](info): set up 2 sockets
Sep 27 18:39:49 dist-test-slave-z47j krb5kdc[22469](info): commencing operation
krb5kdc: starting...
W20260927 18:39:50.191128 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:39:50 dist-test-slave-z47j krb5kdc[22469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:39:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:39:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:39:50.316401 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:46373
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:43031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:46373
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:50.409818 22482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:50.410113 22482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:50.410151 22482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:50.411427 22482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:50.411499 22482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:50.411527 22482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:50.411556 22482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:50.411581 22482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:39:50.413147 22482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:43031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:46373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:46373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:50.413473 22482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:50.413828 22482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:50.419106 22490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:50.419121 22487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:50.419322 22482 server_base.cc:1034] running on GCE node
W20260927 18:39:50.419144 22488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:50.419756 22482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:50.420167 22482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:50.421356 22482 hybrid_clock.cc:648] HybridClock initialized: now 1790534390421305 us; error 65 us; skew 500 ppm
Sep 27 18:39:50 dist-test-slave-z47j krb5kdc[22469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534390, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:50.424346 22482 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:39:50.424901 22482 webserver.cc:460] Webserver started at http://127.11.251.254:40707/ using document root <none> and password file <none>
I20260927 18:39:50.425134 22482 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:50.425182 22482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:50.425292 22482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:50.426644 22482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "965416a6b28d48f6ac3c9697442c1850"
format_stamp: "Formatted at 2026-09-27 18:39:50 on dist-test-slave-z47j"
server_key: "4b4c68d440fc01a449d1fd6c8a48b570"
server_key_iv: "fd9d6cda526cb294eefd7001e6488476"
server_key_version: "encryptionkey@0"
I20260927 18:39:50.427028 22482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "965416a6b28d48f6ac3c9697442c1850"
format_stamp: "Formatted at 2026-09-27 18:39:50 on dist-test-slave-z47j"
server_key: "4b4c68d440fc01a449d1fd6c8a48b570"
server_key_iv: "fd9d6cda526cb294eefd7001e6488476"
server_key_version: "encryptionkey@0"
I20260927 18:39:50.428624 22482 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:39:50.429469 22497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:50.429695 22482 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:39:50.429801 22482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "965416a6b28d48f6ac3c9697442c1850"
format_stamp: "Formatted at 2026-09-27 18:39:50 on dist-test-slave-z47j"
server_key: "4b4c68d440fc01a449d1fd6c8a48b570"
server_key_iv: "fd9d6cda526cb294eefd7001e6488476"
server_key_version: "encryptionkey@0"
I20260927 18:39:50.429878 22482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:50.444442 22482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:50.447005 22482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:50.447139 22482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:50.451793 22482 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:46373
I20260927 18:39:50.451828 22549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:46373 every 8 connection(s)
I20260927 18:39:50.452246 22482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:39:50.453447 22550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:50.456063 22550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850: Bootstrap starting.
I20260927 18:39:50.456669 22550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:50.456943 22550 log.cc:824] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:50.457880 22550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850: No bootstrap required, opened a new log
I20260927 18:39:50.459565 22550 raft_consensus.cc:348] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965416a6b28d48f6ac3c9697442c1850" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46373 } }
I20260927 18:39:50.459721 22550 raft_consensus.cc:374] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:50.459745 22550 raft_consensus.cc:729] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 965416a6b28d48f6ac3c9697442c1850, State: Initialized, Role: FOLLOWER
I20260927 18:39:50.459908 22550 consensus_queue.cc:260] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965416a6b28d48f6ac3c9697442c1850" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46373 } }
I20260927 18:39:50.459997 22550 raft_consensus.cc:388] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:39:50.460022 22550 raft_consensus.cc:482] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:39:50.460053 22550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:50.460952 22550 raft_consensus.cc:504] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965416a6b28d48f6ac3c9697442c1850" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46373 } }
I20260927 18:39:50.461047 22550 leader_election.cc:302] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 965416a6b28d48f6ac3c9697442c1850; no voters:
I20260927 18:39:50.461193 22550 leader_election.cc:288] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:39:50.461261 22553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:50.461474 22553 raft_consensus.cc:686] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850 [term 1 LEADER]: Becoming Leader. State: Replica: 965416a6b28d48f6ac3c9697442c1850, State: Running, Role: LEADER
I20260927 18:39:50.461489 22550 sys_catalog.cc:564] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:50.461658 22553 consensus_queue.cc:237] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965416a6b28d48f6ac3c9697442c1850" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46373 } }
I20260927 18:39:50.462073 22554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "965416a6b28d48f6ac3c9697442c1850" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965416a6b28d48f6ac3c9697442c1850" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46373 } } }
I20260927 18:39:50.462188 22554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:50.462267 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 22482
I20260927 18:39:50.462070 22555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 965416a6b28d48f6ac3c9697442c1850. Latest consensus state: current_term: 1 leader_uuid: "965416a6b28d48f6ac3c9697442c1850" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "965416a6b28d48f6ac3c9697442c1850" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46373 } } }
I20260927 18:39:50.462409 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:39:50.462420 22555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:50.462551 22561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:39:50.462633 12271 external_mini_cluster.cc:1428] Setting key 616642fe6ad62b8e63fbd746a0629f5a
I20260927 18:39:50.463125 22561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:39:50.465183 22561 catalog_manager.cc:1347] Generated new cluster ID: 16744d3323cd47de9649f95e49cba220
I20260927 18:39:50.465242 22561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:39:50 dist-test-slave-z47j krb5kdc[22469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:50.475692 22571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:50.463384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39546 (local address 127.11.251.254:46373)
0927 18:39:50.463891 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:50.463902 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:50.463942 (+ 40us) server_negotiation.cc:407] Connection header received
0927 18:39:50.464083 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:50.464092 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:50.464172 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:50.464331 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:50.465106 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:50.466612 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:50.467709 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:50.468337 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:50.468433 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:50.471644 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:50.471650 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:50.471651 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:50.471667 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:50.473915 (+ 2248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:50.474320 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:50.474324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:50.474324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:50.474445 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:50.474646 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:50.474647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:50.474647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:50.475006 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:50.475173 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:50.475249 (+ 76us) server_negotiation.cc:299] Negotiation successful
0927 18:39:50.475412 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":456,"thread_start_us":63,"threads_started":1}
I20260927 18:39:50.480583 22561 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:39:50.481128 22561 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:39:50.487572 22561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850: Generated new TSK 0
I20260927 18:39:50.487799 22561 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.251.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
I20260927 18:39:50.520648 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:46373
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:43031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:50.613864 22576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:50.614135 22576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:50.614178 22576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:50.615541 22576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:50.615614 22576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:50.615661 22576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:39:50.617192 22576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:43031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:46373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:50.617491 22576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:50.617772 22576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:50.623186 22581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:50.623204 22584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:50.623229 22582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:50.623431 22576 server_base.cc:1034] running on GCE node
I20260927 18:39:50.623696 22576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:50.623968 22576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:50.625125 22576 hybrid_clock.cc:648] HybridClock initialized: now 1790534390625112 us; error 29 us; skew 500 ppm
Sep 27 18:39:50 dist-test-slave-z47j krb5kdc[22469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534390, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:50.632511 22576 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:39:50.633134 22576 webserver.cc:460] Webserver started at http://127.11.251.193:36253/ using document root <none> and password file <none>
I20260927 18:39:50.633407 22576 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:50.633481 22576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:50.633662 22576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:50.634929 22576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "3d780295a92248a68102166fb5ef80bd"
format_stamp: "Formatted at 2026-09-27 18:39:50 on dist-test-slave-z47j"
server_key: "c1d11d87f1323ab1c13c31f4813a0ed9"
server_key_iv: "380d3dad511a22018eb614eab97fb2ea"
server_key_version: "encryptionkey@0"
I20260927 18:39:50.635339 22576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "3d780295a92248a68102166fb5ef80bd"
format_stamp: "Formatted at 2026-09-27 18:39:50 on dist-test-slave-z47j"
server_key: "c1d11d87f1323ab1c13c31f4813a0ed9"
server_key_iv: "380d3dad511a22018eb614eab97fb2ea"
server_key_version: "encryptionkey@0"
I20260927 18:39:50.636814 22576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 18:39:50.637679 22591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:50.637923 22576 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:50.638028 22576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "3d780295a92248a68102166fb5ef80bd"
format_stamp: "Formatted at 2026-09-27 18:39:50 on dist-test-slave-z47j"
server_key: "c1d11d87f1323ab1c13c31f4813a0ed9"
server_key_iv: "380d3dad511a22018eb614eab97fb2ea"
server_key_version: "encryptionkey@0"
I20260927 18:39:50.638115 22576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:50.657040 22576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:50.659672 22576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:50.659880 22576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:50.660152 22576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:50.660507 22576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:50.660571 22576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:50.660619 22576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:50.660650 22576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:50.667706 22576 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:42561
I20260927 18:39:50.667757 22704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:42561 every 8 connection(s)
I20260927 18:39:50.668128 22576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
Sep 27 18:39:50 dist-test-slave-z47j krb5kdc[22469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534390, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:50.677490 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 22576
I20260927 18:39:50.677644 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:39:50.677886 12271 external_mini_cluster.cc:1428] Setting key ebfb37addb18109beb161bdeab1024f3
I20260927 18:39:50.681325 22571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:50.668968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:48971 (local address 127.11.251.254:46373)
0927 18:39:50.669038 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:50.669040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:50.669285 (+ 245us) server_negotiation.cc:407] Connection header received
0927 18:39:50.669879 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:50.669881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:50.669893 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:50.669960 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:50.671053 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:50.671811 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:50.673290 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:50.673648 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:50.673717 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:50.676653 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:50.676656 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:50.676656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:50.676667 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:50.679482 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:50.679967 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:50.679969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:50.679970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:50.680037 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:50.680334 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:50.680335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:50.680336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:50.680521 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:50.680613 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:50.681048 (+ 435us) server_negotiation.cc:299] Negotiation successful
0927 18:39:50.681165 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 18:39:50.681504 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:50.668940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:46373 (local address 127.11.251.193:48971)
0927 18:39:50.669111 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:50.669157 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:50.669753 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:50.670020 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:50.670024 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:50.670668 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:50.670904 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:50.670909 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:50.671930 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:50.671931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:50.673190 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:50.673193 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:50.673724 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:50.673726 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:50.673876 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:50.674523 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:50.674533 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:50.676547 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:50.679559 (+ 3012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:50.679567 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:50.679570 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:50.679894 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:50.680105 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:50.680106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:50.680106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:50.680229 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:50.680601 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:50.680621 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:50.680866 (+ 245us) client_negotiation.cc:769] Sending connection context
0927 18:39:50.681000 (+ 134us) client_negotiation.cc:240] Negotiation successful
0927 18:39:50.681206 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260927 18:39:50.682113 22705 heartbeater.cc:347] Connected to a master server at 127.11.251.254:46373
I20260927 18:39:50.682228 22705 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:50.682454 22705 heartbeater.cc:511] Master 127.11.251.254:46373 requested a full tablet report, sending...
I20260927 18:39:50.683118 22514 ts_manager.cc:194] Registered new tserver with Master: 3d780295a92248a68102166fb5ef80bd (127.11.251.193:42561)
I20260927 18:39:50.683972 22514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:48971
WARNING: no policy specified for kudu/127.11.251.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
I20260927 18:39:50.708906 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:46373
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:43031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:50.799230 22710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:50.799486 22710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:50.799512 22710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:50.800768 22710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:50.800873 22710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:50.800943 22710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:39:50.802584 22710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:43031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:46373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:50.802877 22710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:50.803136 22710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:39:50.808903 22710 server_base.cc:1034] running on GCE node
W20260927 18:39:50.808874 22718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:50.808938 22716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:50.808874 22715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:50.809257 22710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:50.809482 22710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:50.810649 22710 hybrid_clock.cc:648] HybridClock initialized: now 1790534390810625 us; error 50 us; skew 500 ppm
Sep 27 18:39:50 dist-test-slave-z47j krb5kdc[22469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534390, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:50.813709 22710 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:39:50.814286 22710 webserver.cc:460] Webserver started at http://127.11.251.194:39275/ using document root <none> and password file <none>
I20260927 18:39:50.814548 22710 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:50.814618 22710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:50.814829 22710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:50.816092 22710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "5765a9d1d23f490dbb5942c7608b252b"
format_stamp: "Formatted at 2026-09-27 18:39:50 on dist-test-slave-z47j"
server_key: "5f6235268a2a141f5f710d62d34e3e03"
server_key_iv: "ccecdbdd4fa73def9e9f1454cc50fc29"
server_key_version: "encryptionkey@0"
I20260927 18:39:50.816516 22710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "5765a9d1d23f490dbb5942c7608b252b"
format_stamp: "Formatted at 2026-09-27 18:39:50 on dist-test-slave-z47j"
server_key: "5f6235268a2a141f5f710d62d34e3e03"
server_key_iv: "ccecdbdd4fa73def9e9f1454cc50fc29"
server_key_version: "encryptionkey@0"
I20260927 18:39:50.818069 22710 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260927 18:39:50.818943 22725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:50.819166 22710 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:50.819306 22710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "5765a9d1d23f490dbb5942c7608b252b"
format_stamp: "Formatted at 2026-09-27 18:39:50 on dist-test-slave-z47j"
server_key: "5f6235268a2a141f5f710d62d34e3e03"
server_key_iv: "ccecdbdd4fa73def9e9f1454cc50fc29"
server_key_version: "encryptionkey@0"
I20260927 18:39:50.819392 22710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:50.832916 22710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:50.835695 22710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:50.835922 22710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:50.836207 22710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:50.836586 22710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:50.836645 22710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:50.836689 22710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:50.836735 22710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:50.843674 22710 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:37005
I20260927 18:39:50.843720 22838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:37005 every 8 connection(s)
I20260927 18:39:50.844141 22710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:39:50.844900 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 22710
I20260927 18:39:50.845034 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:39:50.845290 12271 external_mini_cluster.cc:1428] Setting key 75481f0ca0003e35755b2748f9641429
Sep 27 18:39:50 dist-test-slave-z47j krb5kdc[22469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534390, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.251.195@KRBTEST.COM; defaulting to no policy
I20260927 18:39:50.856437 22571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:50.845140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:59499 (local address 127.11.251.254:46373)
0927 18:39:50.845235 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:50.845238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:50.845504 (+ 266us) server_negotiation.cc:407] Connection header received
0927 18:39:50.846184 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:50.846185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:50.846205 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:50.846259 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:50.847787 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:50.848398 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:50.849914 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:50.850250 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:50.850330 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:50.853120 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:50.853123 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:50.853123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:50.853132 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:50.854732 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:50.855299 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:50.855300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:50.855301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:50.855373 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:50.855722 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:50.855723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:50.855724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:50.855915 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:50.855977 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:50.856283 (+ 306us) server_negotiation.cc:299] Negotiation successful
0927 18:39:50.856355 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 18:39:50.856657 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:50.845108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:46373 (local address 127.11.251.194:59499)
0927 18:39:50.845343 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:50.845402 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:50.846059 (+ 657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:50.846354 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:50.846360 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:50.847330 (+ 970us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:50.847659 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:50.847667 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:50.848498 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:50.848499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:50.849817 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:50.849821 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:50.850370 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:50.850371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:50.850488 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:50.851185 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:50.851194 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:50.852991 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:50.854822 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:50.854830 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:50.854833 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:50.855192 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:50.855458 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:50.855459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:50.855459 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:50.855609 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:50.855989 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:50.855996 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:50.856157 (+ 161us) client_negotiation.cc:769] Sending connection context
0927 18:39:50.856256 (+ 99us) client_negotiation.cc:240] Negotiation successful
0927 18:39:50.856409 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":123,"threads_started":1}
I20260927 18:39:50.857281 22839 heartbeater.cc:347] Connected to a master server at 127.11.251.254:46373
I20260927 18:39:50.857386 22839 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:50.857700 22839 heartbeater.cc:511] Master 127.11.251.254:46373 requested a full tablet report, sending...
I20260927 18:39:50.858255 22514 ts_manager.cc:194] Registered new tserver with Master: 5765a9d1d23f490dbb5942c7608b252b (127.11.251.194:37005)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.195@KRBTEST.COM" created.
I20260927 18:39:50.858758 22514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:59499
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
I20260927 18:39:50.877835 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:46373
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:43031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:50.968146 22844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:50.968380 22844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:50.968405 22844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:50.969552 22844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:50.969646 22844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:50.969681 22844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:39:50.971165 22844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:43031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:46373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:39:50.971374 22844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:50.971577 22844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:50.977082 22850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:50.977073 22849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:50.977074 22852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:50.977392 22844 server_base.cc:1034] running on GCE node
I20260927 18:39:50.977705 22844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:50.977980 22844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:50.979249 22844 hybrid_clock.cc:648] HybridClock initialized: now 1790534390979230 us; error 29 us; skew 500 ppm
Sep 27 18:39:50 dist-test-slave-z47j krb5kdc[22469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534390, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:50.982338 22844 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:39:50.982873 22844 webserver.cc:460] Webserver started at http://127.11.251.195:45801/ using document root <none> and password file <none>
I20260927 18:39:50.983141 22844 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:50.983219 22844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:50.983393 22844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:39:50.984720 22844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "5dca68ddf09c45f9a836ea629f33dc11"
format_stamp: "Formatted at 2026-09-27 18:39:50 on dist-test-slave-z47j"
server_key: "14d138779f7e5f3e247e9ba1790799de"
server_key_iv: "c4540905a07c5c8577b292c72388e2dc"
server_key_version: "encryptionkey@0"
I20260927 18:39:50.985195 22844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "5dca68ddf09c45f9a836ea629f33dc11"
format_stamp: "Formatted at 2026-09-27 18:39:50 on dist-test-slave-z47j"
server_key: "14d138779f7e5f3e247e9ba1790799de"
server_key_iv: "c4540905a07c5c8577b292c72388e2dc"
server_key_version: "encryptionkey@0"
I20260927 18:39:50.986819 22844 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260927 18:39:50.987749 22859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:50.987954 22844 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:39:50.988056 22844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "5dca68ddf09c45f9a836ea629f33dc11"
format_stamp: "Formatted at 2026-09-27 18:39:50 on dist-test-slave-z47j"
server_key: "14d138779f7e5f3e247e9ba1790799de"
server_key_iv: "c4540905a07c5c8577b292c72388e2dc"
server_key_version: "encryptionkey@0"
I20260927 18:39:50.988133 22844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:51.006443 22844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:51.008620 22844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:51.008805 22844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:51.009087 22844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:51.009449 22844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:51.009505 22844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:51.009552 22844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:51.009599 22844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:51.016721 22844 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:39257
I20260927 18:39:51.016765 22972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:39257 every 8 connection(s)
I20260927 18:39:51.017144 22844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:39:51.024132 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 22844
I20260927 18:39:51.024243 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
Sep 27 18:39:51 dist-test-slave-z47j krb5kdc[22469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534390, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:39:51.024471 12271 external_mini_cluster.cc:1428] Setting key 3efb125db55475140e54b18b532db3f4
I20260927 18:39:51.028143 22571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:51.017978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:38745 (local address 127.11.251.254:46373)
0927 18:39:51.018058 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:51.018061 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:51.018302 (+ 241us) server_negotiation.cc:407] Connection header received
0927 18:39:51.018957 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:51.018959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:51.018971 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:51.019043 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:51.020150 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.020708 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.021859 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.022070 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.022129 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:51.024956 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:51.024958 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:51.024958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:51.024966 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:51.026722 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:51.027203 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:51.027203 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:51.027204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:51.027260 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:51.027512 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:51.027512 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:51.027513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:51.027634 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:51.027687 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:51.027994 (+ 307us) server_negotiation.cc:299] Negotiation successful
0927 18:39:51.028067 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 18:39:51.028367 22975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:51.017956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:46373 (local address 127.11.251.195:38745)
0927 18:39:51.018154 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:51.018201 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:51.018785 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:51.019128 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:51.019132 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:51.019788 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:51.020031 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:51.020036 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.020792 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.020793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:51.021781 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:51.021784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.022146 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.022146 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:51.022258 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:51.022931 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:51.022941 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:51.024828 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:51.026862 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:51.026869 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:51.026871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:51.027124 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:51.027326 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:51.027327 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:51.027328 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:51.027449 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:51.027689 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:51.027694 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:51.027866 (+ 172us) client_negotiation.cc:769] Sending connection context
0927 18:39:51.027961 (+ 95us) client_negotiation.cc:240] Negotiation successful
0927 18:39:51.028114 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":118,"threads_started":1}
I20260927 18:39:51.028879 22973 heartbeater.cc:347] Connected to a master server at 127.11.251.254:46373
I20260927 18:39:51.029002 22973 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:51.029206 22973 heartbeater.cc:511] Master 127.11.251.254:46373 requested a full tablet report, sending...
I20260927 18:39:51.029685 22514 ts_manager.cc:194] Registered new tserver with Master: 5dca68ddf09c45f9a836ea629f33dc11 (127.11.251.195:39257)
I20260927 18:39:51.030112 22514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:38745
I20260927 18:39:51.036783 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:39:51.045178 22571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:51.038407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39560 (local address 127.11.251.254:46373)
0927 18:39:51.038511 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:51.038514 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:51.038619 (+ 105us) server_negotiation.cc:407] Connection header received
0927 18:39:51.038715 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:51.038715 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:51.038727 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:51.038796 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:51.039410 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.039943 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.040843 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.041051 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.041140 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:51.042611 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:51.042612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:51.042612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:51.042620 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:51.043919 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:51.044288 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:51.044289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:51.044290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:51.044358 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:51.044564 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:51.044565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:51.044565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:51.044750 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:51.044872 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:51.044971 (+ 99us) server_negotiation.cc:299] Negotiation successful
0927 18:39:51.045061 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 18:39:51.046850 22514 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39560:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:39:51.048851 22514 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:39:51.056636 22984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:51.051931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:42561 (local address 127.0.0.1:50552)
0927 18:39:51.052077 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:51.052129 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:51.052210 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:51.052591 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:51.052595 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:51.052599 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:51.052693 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:51.052696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.054310 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.054311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:51.055301 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:51.055304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.056285 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.056285 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:51.056369 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:51.056386 (+ 17us) client_negotiation.cc:769] Sending connection context
0927 18:39:51.056465 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 18:39:51.056526 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":110,"thread_start_us":80,"threads_started":1}
I20260927 18:39:51.056636 22987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:51.052174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:37005 (local address 127.0.0.1:54876)
0927 18:39:51.052335 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:51.052345 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:51.052559 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:51.052953 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:51.052954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:51.052956 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:51.052994 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:51.052995 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.054306 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.054307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:51.055464 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:51.055468 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.056369 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.056369 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:51.056447 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:51.056451 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:39:51.056499 (+ 48us) client_negotiation.cc:240] Negotiation successful
0927 18:39:51.056528 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":3,"thread_start_us":71,"threads_started":1}
I20260927 18:39:51.056644 22986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:51.052060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50552 (local address 127.11.251.193:42561)
0927 18:39:51.052372 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:51.052377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:51.052388 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:39:51.052435 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:51.052438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:51.052456 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:51.052779 (+ 323us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:51.052876 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.054173 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.055411 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.056196 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.056284 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:51.056352 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:51.056492 (+ 140us) server_negotiation.cc:299] Negotiation successful
0927 18:39:51.056547 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":89,"threads_started":1}
I20260927 18:39:51.057525 22989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:51.052411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54876 (local address 127.11.251.194:37005)
0927 18:39:51.052765 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:51.052769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:51.052779 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:39:51.052824 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:51.052826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:51.052842 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:51.052954 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:51.053050 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.054181 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.055564 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.056289 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.056358 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:51.056405 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:51.057352 (+ 947us) server_negotiation.cc:299] Negotiation successful
0927 18:39:51.057430 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":95,"threads_started":1}
I20260927 18:39:51.057556 22985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:51.051941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:39257 (local address 127.0.0.1:55048)
0927 18:39:51.052112 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:51.052129 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:51.052210 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:51.052841 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:51.052842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:51.052844 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:51.052897 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:51.052898 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.053970 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.053971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:51.055488 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:51.055490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.057305 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.057306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:51.057373 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:51.057377 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:39:51.057437 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 18:39:51.057480 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":190,"thread_start_us":106,"threads_started":1}
I20260927 18:39:51.057603 22988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:51.052280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55048 (local address 127.11.251.195:39257)
0927 18:39:51.052658 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:51.052663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:51.052675 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:39:51.052719 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:51.052721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:51.052737 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:51.052843 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:51.053070 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.053859 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.056471 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.057238 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.057308 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:51.057357 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:51.057452 (+ 95us) server_negotiation.cc:299] Negotiation successful
0927 18:39:51.057497 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":131,"threads_started":1}
I20260927 18:39:51.057610 22639 tablet_service.cc:1467] Processing CreateTablet for tablet 1cb085ef065e46f68a402ad2352c98b4 (DEFAULT_TABLE table=test-table [id=d8485482a88049018f192ddec112ff8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:51.058113 22639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cb085ef065e46f68a402ad2352c98b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:51.058275 22907 tablet_service.cc:1467] Processing CreateTablet for tablet 1cb085ef065e46f68a402ad2352c98b4 (DEFAULT_TABLE table=test-table [id=d8485482a88049018f192ddec112ff8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:51.058609 22907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cb085ef065e46f68a402ad2352c98b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:51.058548 22773 tablet_service.cc:1467] Processing CreateTablet for tablet 1cb085ef065e46f68a402ad2352c98b4 (DEFAULT_TABLE table=test-table [id=d8485482a88049018f192ddec112ff8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:51.058864 22773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cb085ef065e46f68a402ad2352c98b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:51.061758 22990 tablet_bootstrap.cc:492] T 1cb085ef065e46f68a402ad2352c98b4 P 5dca68ddf09c45f9a836ea629f33dc11: Bootstrap starting.
I20260927 18:39:51.062077 22991 tablet_bootstrap.cc:492] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd: Bootstrap starting.
I20260927 18:39:51.062357 22992 tablet_bootstrap.cc:492] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b: Bootstrap starting.
I20260927 18:39:51.062733 22990 tablet_bootstrap.cc:654] T 1cb085ef065e46f68a402ad2352c98b4 P 5dca68ddf09c45f9a836ea629f33dc11: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:51.062839 22991 tablet_bootstrap.cc:654] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:51.063023 22992 tablet_bootstrap.cc:654] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:51.063477 22991 log.cc:824] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:51.063575 22990 log.cc:824] T 1cb085ef065e46f68a402ad2352c98b4 P 5dca68ddf09c45f9a836ea629f33dc11: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:51.063701 22992 log.cc:824] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:51.064291 22991 tablet_bootstrap.cc:492] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd: No bootstrap required, opened a new log
I20260927 18:39:51.064318 22990 tablet_bootstrap.cc:492] T 1cb085ef065e46f68a402ad2352c98b4 P 5dca68ddf09c45f9a836ea629f33dc11: No bootstrap required, opened a new log
I20260927 18:39:51.064376 22990 ts_tablet_manager.cc:1397] T 1cb085ef065e46f68a402ad2352c98b4 P 5dca68ddf09c45f9a836ea629f33dc11: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:39:51.064384 22991 ts_tablet_manager.cc:1397] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:39:51.064471 22992 tablet_bootstrap.cc:492] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b: No bootstrap required, opened a new log
I20260927 18:39:51.064536 22992 ts_tablet_manager.cc:1397] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:39:51.065948 22991 raft_consensus.cc:348] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dca68ddf09c45f9a836ea629f33dc11" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 39257 } } peers { permanent_uuid: "5765a9d1d23f490dbb5942c7608b252b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37005 } } peers { permanent_uuid: "3d780295a92248a68102166fb5ef80bd" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42561 } }
I20260927 18:39:51.066120 22991 raft_consensus.cc:374] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:51.066164 22991 raft_consensus.cc:729] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d780295a92248a68102166fb5ef80bd, State: Initialized, Role: FOLLOWER
I20260927 18:39:51.066283 22991 consensus_queue.cc:260] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dca68ddf09c45f9a836ea629f33dc11" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 39257 } } peers { permanent_uuid: "5765a9d1d23f490dbb5942c7608b252b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37005 } } peers { permanent_uuid: "3d780295a92248a68102166fb5ef80bd" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42561 } }
I20260927 18:39:51.066591 22991 ts_tablet_manager.cc:1428] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:39:51.066521 22990 raft_consensus.cc:348] T 1cb085ef065e46f68a402ad2352c98b4 P 5dca68ddf09c45f9a836ea629f33dc11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dca68ddf09c45f9a836ea629f33dc11" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 39257 } } peers { permanent_uuid: "5765a9d1d23f490dbb5942c7608b252b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37005 } } peers { permanent_uuid: "3d780295a92248a68102166fb5ef80bd" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42561 } }
I20260927 18:39:51.066567 22992 raft_consensus.cc:348] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dca68ddf09c45f9a836ea629f33dc11" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 39257 } } peers { permanent_uuid: "5765a9d1d23f490dbb5942c7608b252b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37005 } } peers { permanent_uuid: "3d780295a92248a68102166fb5ef80bd" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42561 } }
I20260927 18:39:51.066666 22990 raft_consensus.cc:374] T 1cb085ef065e46f68a402ad2352c98b4 P 5dca68ddf09c45f9a836ea629f33dc11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:51.066694 22992 raft_consensus.cc:374] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:51.066704 22990 raft_consensus.cc:729] T 1cb085ef065e46f68a402ad2352c98b4 P 5dca68ddf09c45f9a836ea629f33dc11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dca68ddf09c45f9a836ea629f33dc11, State: Initialized, Role: FOLLOWER
I20260927 18:39:51.066740 22992 raft_consensus.cc:729] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5765a9d1d23f490dbb5942c7608b252b, State: Initialized, Role: FOLLOWER
I20260927 18:39:51.066766 22705 heartbeater.cc:503] Master 127.11.251.254:46373 was elected leader, sending a full tablet report...
I20260927 18:39:51.066810 22990 consensus_queue.cc:260] T 1cb085ef065e46f68a402ad2352c98b4 P 5dca68ddf09c45f9a836ea629f33dc11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dca68ddf09c45f9a836ea629f33dc11" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 39257 } } peers { permanent_uuid: "5765a9d1d23f490dbb5942c7608b252b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37005 } } peers { permanent_uuid: "3d780295a92248a68102166fb5ef80bd" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42561 } }
I20260927 18:39:51.066869 22992 consensus_queue.cc:260] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dca68ddf09c45f9a836ea629f33dc11" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 39257 } } peers { permanent_uuid: "5765a9d1d23f490dbb5942c7608b252b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37005 } } peers { permanent_uuid: "3d780295a92248a68102166fb5ef80bd" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42561 } }
I20260927 18:39:51.067093 22990 ts_tablet_manager.cc:1428] T 1cb085ef065e46f68a402ad2352c98b4 P 5dca68ddf09c45f9a836ea629f33dc11: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:39:51.067137 22992 ts_tablet_manager.cc:1428] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:39:51.067255 22839 heartbeater.cc:503] Master 127.11.251.254:46373 was elected leader, sending a full tablet report...
I20260927 18:39:51.067494 22973 heartbeater.cc:503] Master 127.11.251.254:46373 was elected leader, sending a full tablet report...
W20260927 18:39:51.094797 22840 tablet.cc:2369] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:39:51.123486 22996 raft_consensus.cc:482] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:39:51.123661 22996 raft_consensus.cc:504] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dca68ddf09c45f9a836ea629f33dc11" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 39257 } } peers { permanent_uuid: "5765a9d1d23f490dbb5942c7608b252b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37005 } } peers { permanent_uuid: "3d780295a92248a68102166fb5ef80bd" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42561 } }
I20260927 18:39:51.124107 22996 leader_election.cc:288] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5dca68ddf09c45f9a836ea629f33dc11 (127.11.251.195:39257), 5765a9d1d23f490dbb5942c7608b252b (127.11.251.194:37005)
I20260927 18:39:51.127777 22989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:51.124797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:39729 (local address 127.11.251.194:37005)
0927 18:39:51.124871 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:51.124873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:51.124883 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:39:51.124920 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:51.124921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:51.124930 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:51.125007 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:51.125151 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.125877 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.126942 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.127361 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.127463 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:51.127475 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:51.127643 (+ 168us) server_negotiation.cc:299] Negotiation successful
0927 18:39:51.127692 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:39:51.127791 22999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:51.124533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:37005 (local address 127.11.251.193:39729)
0927 18:39:51.124728 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:51.124739 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:51.124801 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:51.125011 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:51.125011 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:51.125013 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:51.125083 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:51.125084 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.125972 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.125973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:51.126807 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:51.126809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.127464 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.127465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:51.127554 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:51.127559 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:39:51.127622 (+ 63us) client_negotiation.cc:240] Negotiation successful
0927 18:39:51.127670 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":1,"thread_start_us":140,"threads_started":1}
I20260927 18:39:51.127820 22988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:51.124456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:58917 (local address 127.11.251.195:39257)
0927 18:39:51.124525 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:51.124527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:51.124541 (+ 14us) server_negotiation.cc:407] Connection header received
0927 18:39:51.124626 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:51.124627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:51.124644 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:51.124710 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:51.124938 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.125851 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.126921 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.127378 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.127477 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:51.127487 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:51.127682 (+ 195us) server_negotiation.cc:299] Negotiation successful
0927 18:39:51.127728 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:39:51.127808 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:51.124363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:39257 (local address 127.11.251.193:58917)
0927 18:39:51.124445 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:51.124459 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:51.124551 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:51.124714 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:51.124715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:51.124717 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:51.124809 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:51.124831 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.125933 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.125934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:51.126804 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:51.126807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.127485 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.127487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:51.127587 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:51.127590 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:39:51.127665 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 18:39:51.127713 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":3}
I20260927 18:39:51.128281 22927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cb085ef065e46f68a402ad2352c98b4" candidate_uuid: "3d780295a92248a68102166fb5ef80bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dca68ddf09c45f9a836ea629f33dc11" is_pre_election: true
I20260927 18:39:51.128337 22793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cb085ef065e46f68a402ad2352c98b4" candidate_uuid: "3d780295a92248a68102166fb5ef80bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5765a9d1d23f490dbb5942c7608b252b" is_pre_election: true
I20260927 18:39:51.128518 22793 raft_consensus.cc:2454] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d780295a92248a68102166fb5ef80bd in term 0.
I20260927 18:39:51.128520 22927 raft_consensus.cc:2454] T 1cb085ef065e46f68a402ad2352c98b4 P 5dca68ddf09c45f9a836ea629f33dc11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d780295a92248a68102166fb5ef80bd in term 0.
I20260927 18:39:51.128774 22594 leader_election.cc:302] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d780295a92248a68102166fb5ef80bd, 5765a9d1d23f490dbb5942c7608b252b; no voters:
I20260927 18:39:51.128959 22996 raft_consensus.cc:2781] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:39:51.129038 22996 raft_consensus.cc:482] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:39:51.129088 22996 raft_consensus.cc:3037] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:51.130052 22996 raft_consensus.cc:504] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dca68ddf09c45f9a836ea629f33dc11" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 39257 } } peers { permanent_uuid: "5765a9d1d23f490dbb5942c7608b252b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37005 } } peers { permanent_uuid: "3d780295a92248a68102166fb5ef80bd" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42561 } }
I20260927 18:39:51.130211 22996 leader_election.cc:288] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd [CANDIDATE]: Term 1 election: Requested vote from peers 5dca68ddf09c45f9a836ea629f33dc11 (127.11.251.195:39257), 5765a9d1d23f490dbb5942c7608b252b (127.11.251.194:37005)
I20260927 18:39:51.130420 22793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cb085ef065e46f68a402ad2352c98b4" candidate_uuid: "3d780295a92248a68102166fb5ef80bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5765a9d1d23f490dbb5942c7608b252b"
I20260927 18:39:51.130440 22927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cb085ef065e46f68a402ad2352c98b4" candidate_uuid: "3d780295a92248a68102166fb5ef80bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dca68ddf09c45f9a836ea629f33dc11"
I20260927 18:39:51.130542 22793 raft_consensus.cc:3037] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:51.130568 22927 raft_consensus.cc:3037] T 1cb085ef065e46f68a402ad2352c98b4 P 5dca68ddf09c45f9a836ea629f33dc11 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:51.131670 22927 raft_consensus.cc:2454] T 1cb085ef065e46f68a402ad2352c98b4 P 5dca68ddf09c45f9a836ea629f33dc11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d780295a92248a68102166fb5ef80bd in term 1.
I20260927 18:39:51.131670 22793 raft_consensus.cc:2454] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d780295a92248a68102166fb5ef80bd in term 1.
I20260927 18:39:51.131867 22594 leader_election.cc:302] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d780295a92248a68102166fb5ef80bd, 5765a9d1d23f490dbb5942c7608b252b; no voters:
I20260927 18:39:51.131970 22996 raft_consensus.cc:2781] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:51.132164 22996 raft_consensus.cc:686] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd [term 1 LEADER]: Becoming Leader. State: Replica: 3d780295a92248a68102166fb5ef80bd, State: Running, Role: LEADER
I20260927 18:39:51.132287 22996 consensus_queue.cc:237] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dca68ddf09c45f9a836ea629f33dc11" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 39257 } } peers { permanent_uuid: "5765a9d1d23f490dbb5942c7608b252b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37005 } } peers { permanent_uuid: "3d780295a92248a68102166fb5ef80bd" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42561 } }
I20260927 18:39:51.133383 22513 catalog_manager.cc:5495] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d780295a92248a68102166fb5ef80bd (127.11.251.193). New cstate: current_term: 1 leader_uuid: "3d780295a92248a68102166fb5ef80bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dca68ddf09c45f9a836ea629f33dc11" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 39257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5765a9d1d23f490dbb5942c7608b252b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d780295a92248a68102166fb5ef80bd" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 42561 } health_report { overall_health: HEALTHY } } }
I20260927 18:39:51.148837 22986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:51.146338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50562 (local address 127.11.251.193:42561)
0927 18:39:51.146404 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:51.146406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:51.146414 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:39:51.146495 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:51.146496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:51.146514 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:51.146572 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:39:51.146701 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.147205 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.148047 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:51.148236 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:51.148288 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:51.148460 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:39:51.148582 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:39:51.148648 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:51.148680 (+ 32us) server_negotiation.cc:299] Negotiation successful
0927 18:39:51.148700 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:39:51.150987 22793 raft_consensus.cc:1261] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b [term 1 FOLLOWER]: Refusing update from remote peer 3d780295a92248a68102166fb5ef80bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:51.151013 22927 raft_consensus.cc:1261] T 1cb085ef065e46f68a402ad2352c98b4 P 5dca68ddf09c45f9a836ea629f33dc11 [term 1 FOLLOWER]: Refusing update from remote peer 3d780295a92248a68102166fb5ef80bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:51.151425 23000 consensus_queue.cc:1035] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "5765a9d1d23f490dbb5942c7608b252b" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:51.151579 22996 consensus_queue.cc:1035] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dca68ddf09c45f9a836ea629f33dc11" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 39257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:51.154883 23004 mvcc.cc:205] Tried to move back new op lower bound from 7334028866151346176 to 7334028866078842880. Current Snapshot: MvccSnapshot[applied={T|T < 7334028866151346176}]
I20260927 18:39:51.155030 23005 mvcc.cc:205] Tried to move back new op lower bound from 7334028866151346176 to 7334028866078842880. Current Snapshot: MvccSnapshot[applied={T|T < 7334028866151346176}]
I20260927 18:39:51.156821 22513 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:51.156945 22513 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:51.157843 22513 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 965416a6b28d48f6ac3c9697442c1850: Sending DeleteTablet for 3 replicas of tablet 1cb085ef065e46f68a402ad2352c98b4
I20260927 18:39:51.158241 22907 tablet_service.cc:1514] Processing DeleteTablet for tablet 1cb085ef065e46f68a402ad2352c98b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:51 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:55048
I20260927 18:39:51.158269 22639 tablet_service.cc:1514] Processing DeleteTablet for tablet 1cb085ef065e46f68a402ad2352c98b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:51 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:50552
I20260927 18:39:51.158488 23013 tablet_replica.cc:330] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd: stopping tablet replica
I20260927 18:39:51.158509 22773 tablet_service.cc:1514] Processing DeleteTablet for tablet 1cb085ef065e46f68a402ad2352c98b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:51 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:54876
I20260927 18:39:51.158595 23013 raft_consensus.cc:2229] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:39:51.158707 23013 raft_consensus.cc:2258] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:51.158809 23012 tablet_replica.cc:330] T 1cb085ef065e46f68a402ad2352c98b4 P 5dca68ddf09c45f9a836ea629f33dc11: stopping tablet replica
I20260927 18:39:51.158897 23012 raft_consensus.cc:2229] T 1cb085ef065e46f68a402ad2352c98b4 P 5dca68ddf09c45f9a836ea629f33dc11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:51.158993 23012 raft_consensus.cc:2258] T 1cb085ef065e46f68a402ad2352c98b4 P 5dca68ddf09c45f9a836ea629f33dc11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:51.158980 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 22576
I20260927 18:39:51.159144 23014 tablet_replica.cc:330] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b: stopping tablet replica
I20260927 18:39:51.159225 23014 raft_consensus.cc:2229] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:51.159219 23013 ts_tablet_manager.cc:1905] T 1cb085ef065e46f68a402ad2352c98b4 P 3d780295a92248a68102166fb5ef80bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:51.159312 23014 raft_consensus.cc:2258] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:51.159960 23014 ts_tablet_manager.cc:1905] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:51.160455 23012 ts_tablet_manager.cc:1905] T 1cb085ef065e46f68a402ad2352c98b4 P 5dca68ddf09c45f9a836ea629f33dc11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:51.162060 23014 ts_tablet_manager.cc:1918] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:39:51.162132 23014 log.cc:1196] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/1cb085ef065e46f68a402ad2352c98b4
I20260927 18:39:51.162315 23012 ts_tablet_manager.cc:1918] T 1cb085ef065e46f68a402ad2352c98b4 P 5dca68ddf09c45f9a836ea629f33dc11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:39:51.162375 23012 log.cc:1196] T 1cb085ef065e46f68a402ad2352c98b4 P 5dca68ddf09c45f9a836ea629f33dc11: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/1cb085ef065e46f68a402ad2352c98b4
I20260927 18:39:51.162473 23014 ts_tablet_manager.cc:1939] T 1cb085ef065e46f68a402ad2352c98b4 P 5765a9d1d23f490dbb5942c7608b252b: Deleting consensus metadata
I20260927 18:39:51.162694 23012 ts_tablet_manager.cc:1939] T 1cb085ef065e46f68a402ad2352c98b4 P 5dca68ddf09c45f9a836ea629f33dc11: Deleting consensus metadata
I20260927 18:39:51.163033 22500 catalog_manager.cc:4841] TS 5765a9d1d23f490dbb5942c7608b252b (127.11.251.194:37005): tablet 1cb085ef065e46f68a402ad2352c98b4 (table test-table [id=d8485482a88049018f192ddec112ff8c]) successfully deleted
I20260927 18:39:51.163318 22500 catalog_manager.cc:4841] TS 5dca68ddf09c45f9a836ea629f33dc11 (127.11.251.195:39257): tablet 1cb085ef065e46f68a402ad2352c98b4 (table test-table [id=d8485482a88049018f192ddec112ff8c]) successfully deleted
W20260927 18:39:51.166777 22499 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.251.193:42561 (error 108)
I20260927 18:39:51.167090 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 22710
I20260927 18:39:51.167733 22985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:51.167209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:42561 (local address 127.0.0.1:50578)
0927 18:39:51.167649 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:51.167695 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.193:42561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411}
W20260927 18:39:51.167873 22499 catalog_manager.cc:4584] TS 3d780295a92248a68102166fb5ef80bd (127.11.251.193:42561): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1cb085ef065e46f68a402ad2352c98b4: Network error: Client connection negotiation failed: client connection to 127.11.251.193:42561: connect: Connection refused (error 111)
I20260927 18:39:51.173377 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 22844
I20260927 18:39:51.180788 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 22482
2026-09-27T18:39:51Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2044 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:39:51 dist-test-slave-z47j krb5kdc[23018](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:39:51 dist-test-slave-z47j krb5kdc[23018](info): set up 2 sockets
Sep 27 18:39:51 dist-test-slave-z47j krb5kdc[23018](info): commencing operation
krb5kdc: starting...
W20260927 18:39:52.243223 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:39:52 dist-test-slave-z47j krb5kdc[23018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:39:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:39:52Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.251.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.251.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.251.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/rangeradmin_127.11.251.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.251.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/rangeradmin_127.11.251.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.251.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.251.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.251.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/rangerlookup_127.11.251.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.251.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/rangerlookup_127.11.251.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.251.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.251.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.251.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/HTTP_127.11.251.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.251.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/HTTP_127.11.251.212@KRBTEST.COM.keytab.
I20260927 18:39:52.428956 12271 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskbSvNM2/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/postgres -l logfile start
2026-09-27 18:39:55.219 UTC [23048] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-27 18:39:55.219 UTC [23048] LOG: listening on IPv4 address "127.11.251.212", port 54691
2026-09-27 18:39:55.222 UTC [23048] LOG: listening on Unix socket "/tmp/.s.PGSQL.54691"
2026-09-27 18:39:55.245 UTC [23051] LOG: database system was shut down at 2026-09-27 18:39:53 UTC
2026-09-27 18:39:55.248 UTC [23048] LOG: database system is ready to accept connections
I20260927 18:39:56.226773 12271 mini_postgres.cc:96] Postgres bound to 54691
2026-09-27 18:39:56.233 UTC [23059] FATAL: database "slave" does not exist
127.11.251.212:54691 - accepting connections
I20260927 18:39:56.234514 12271 mini_ranger.cc:161] Starting Ranger...
I20260927 18:39:56.250306 12271 mini_ranger.cc:84] Created miniranger Postgres user
I20260927 18:39:56.790494 12271 mini_ranger.cc:87] Created ranger Postgres database
I20260927 18:39:56.790617 12271 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ranger-admin
2026-09-27 18:39:57,091 [I] DB FLAVOR :POSTGRES
2026-09-27 18:39:57,092 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 18:39:57,092 [I] Checking connection..
2026-09-27 18:39:57,092 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-27 18:39:57,362 [I] Checking connection passed.
2026-09-27 18:39:57,363 [I] --------- Verifying version history table ---------
2026-09-27 18:39:57,363 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 18:39:57,623 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 18:39:57,623 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 18:39:57,889 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 18:39:57,889 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-27 18:39:57,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-27 18:39:58.146 UTC [23123] WARNING: there is no transaction in progress
2026-09-27 18:39:58,153 [I] create_dbversion_catalog.sql file imported successfully
2026-09-27 18:39:58,153 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 18:39:58,440 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-27 18:39:58,440 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 18:39:58,441 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-27 18:39:58,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-27 18:39:58,977 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-z47j.c.gcp-upstream.internal','N') ;"
2026-09-27 18:39:59,226 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 18:39:59,508 [I] Table x_portal_user does not exist in database ranger
2026-09-27 18:39:59,509 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 18:39:59,798 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-27 18:39:59,798 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 18:40:00,068 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 18:40:00,367 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-27 18:40:00,368 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-27 18:40:01.449 UTC [23231] WARNING: there is no transaction in progress
2026-09-27 18:40:01.468 UTC [23231] WARNING: there is no transaction in progress
2026-09-27 18:40:01.487 UTC [23231] WARNING: there is no transaction in progress
2026-09-27 18:40:01.506 UTC [23231] WARNING: there is no transaction in progress
2026-09-27 18:40:01.533 UTC [23231] WARNING: there is no transaction in progress
2026-09-27 18:40:01.556 UTC [23231] WARNING: there is no transaction in progress
2026-09-27 18:40:01.618 UTC [23231] WARNING: there is no transaction in progress
2026-09-27 18:40:01.626 UTC [23231] WARNING: there is no transaction in progress
2026-09-27 18:40:01.642 UTC [23231] WARNING: there is no transaction in progress
2026-09-27 18:40:01.652 UTC [23231] WARNING: there is no transaction in progress
2026-09-27 18:40:01.663 UTC [23231] WARNING: there is no transaction in progress
2026-09-27 18:40:01.678 UTC [23231] WARNING: there is no transaction in progress
2026-09-27 18:40:01.689 UTC [23231] WARNING: there is no transaction in progress
2026-09-27 18:40:01.696 UTC [23231] WARNING: there is no transaction in progress
2026-09-27 18:40:01.704 UTC [23231] WARNING: there is no transaction in progress
2026-09-27 18:40:01.710 UTC [23231] WARNING: there is no transaction in progress
2026-09-27 18:40:02,428 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-27 18:40:02,428 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-27 18:40:02,691 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-27 18:40:02,692 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 18:40:02,956 [I] Table x_portal_user already exists in database 'ranger'
2026-09-27 18:40:02,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 18:40:03,245 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-27 18:40:03,246 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 18:40:03,523 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 18:40:03,787 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-z47j.c.gcp-upstream.internal';"
2026-09-27 18:40:04,031 [I] CORE_DB_SCHEMA import status has been updated
2026-09-27 18:40:04,032 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-27 18:40:04,282 [I] DB_PATCHES have already been applied
I20260927 18:40:04.289255 12271 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ranger-admin:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskbSvNM2/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260927 18:40:04.289341 12271 mini_ranger.cc:193] Using host: 127.11.251.212
I20260927 18:40:04.293083 12271 mini_ranger.cc:227] Ranger admin URL: http://127.11.251.212:33807
Sep 27, 2026 6:40:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 6:40:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 6:40:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ranger-admin/ews/webapp
Sep 27, 2026 6:40:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 27, 2026 6:40:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 27, 2026 6:40:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ranger-admin/ews/webapp].
Sep 27, 2026 6:40:05 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260927 18:40:05.445515 12271 mini_ranger.cc:160] Time spent starting Ranger: real 9.211s user 0.004s sys 0.007s
Sep 27, 2026 6:40:28 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 27, 2026 6:40:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 27, 2026 6:40:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 27, 2026 6:40:28 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 27, 2026 6:40:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 27, 2026 6:40:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 27, 2026 6:40:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 27, 2026 6:40:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 27, 2026 6:40:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 27, 2026 6:40:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 27, 2026 6:40:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 27, 2026 6:40:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 27, 2026 6:40:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 27, 2026 6:40:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 27, 2026 6:40:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 27, 2026 6:40:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 27, 2026 6:40:28 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 27, 2026 6:40:29 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260927 18:40:29.783105 12271 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.251.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.251.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.251.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/rangerkms_127.11.251.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.251.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/rangerkms_127.11.251.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.251.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.251.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.251.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/HTTP_127.11.251.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.251.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/HTTP_127.11.251.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 27 18:40:29 dist-test-slave-z47j krb5kdc[23018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534429, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260927 18:40:29.890862 12271 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260927 18:40:29.901332 12271 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260927 18:40:30.831961 12271 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260927 18:40:30.832049 12271 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ranger-kms
I20260927 18:40:30.832065 12271 mini_ranger_kms.cc:227] Using postgres at 127.11.251.212:54691
2026-09-27 18:40:31,049 [I] DB FLAVOR :POSTGRES
2026-09-27 18:40:31,049 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 18:40:31,050 [I] Checking connection
2026-09-27 18:40:31,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-27 18:40:31,346 [I] connection success
2026-09-27 18:40:31,347 [I] --------- Verifying Ranger DB tables ---------
2026-09-27 18:40:31,347 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-27 18:40:31,347 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-27 18:40:31,623 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-27 18:40:31,624 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 18:40:31,624 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-27 18:40:31,624 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:54691/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-27 18:40:31,902 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260927 18:40:32.426879 12271 mini_ranger_kms.cc:310] Created kms service
I20260927 18:40:32.690402 12271 mini_ranger_kms.cc:326] Created kudu user
I20260927 18:40:32.738292 12271 mini_ranger_kms.cc:343] Created rangerkms user
Sep 27 18:40:32 dist-test-slave-z47j krb5kdc[23018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534429, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
I20260927 18:40:33.180858 12271 mini_ranger_kms.cc:384] Added ranger policy
I20260927 18:40:33.181015 12271 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ranger-kms:/tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260927 18:40:33.181039 12271 mini_ranger_kms.cc:240] Using host: 127.11.251.212
I20260927 18:40:33.186663 12271 mini_ranger_kms.cc:276] Ranger KMS PID: 23412
I20260927 18:40:33.186803 12271 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.251.212:33271
Sep 27, 2026 6:40:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 6:40:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 6:40:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 27, 2026 6:40:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 27, 2026 6:40:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ranger-kms/ews/webapp].
Sep 27, 2026 6:40:35 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260927 18:40:35.474252 12271 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.583s user 0.009s sys 0.188s
I20260927 18:40:35.474377 12271 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260927 18:40:35.474469 12271 mini_ranger_kms.cc:401] 127.11.251.212:33271/kms/v1/keys
Sep 27 18:40:35 dist-test-slave-z47j krb5kdc[23018](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790534435, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.251.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 27 18:40:36 dist-test-slave-z47j krb5kdc[23018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534435, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.251.212@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
Sep 27 18:40:36 dist-test-slave-z47j krb5kdc[23018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534435, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.251.212@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
Sep 27 18:40:39 dist-test-slave-z47j krb5kdc[23018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:40:39.232767 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:38531
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:37173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:38531
--ranger_config_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.251.212:33271/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:40:39.341717 23462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:39.341953 23462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:39.341976 23462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:39.343292 23462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:40:39.343335 23462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:39.343349 23462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:39.343362 23462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:40:39.343375 23462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:40:39.345013 23462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:37173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.251.212:33271/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:38531
--ranger_config_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:38531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:40:39.345299 23462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:39.345619 23462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:39.351797 23470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:39.351797 23468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:39.351837 23467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:39.352144 23462 server_base.cc:1034] running on GCE node
I20260927 18:40:39.352432 23462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:39.352780 23462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:39.353986 23462 hybrid_clock.cc:648] HybridClock initialized: now 1790534439353961 us; error 44 us; skew 500 ppm
Sep 27 18:40:39 dist-test-slave-z47j krb5kdc[23018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534439, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:39.357003 23462 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:40:39.357507 23462 webserver.cc:460] Webserver started at http://127.11.251.254:33883/ using document root <none> and password file <none>
I20260927 18:40:39.357790 23462 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:39.357856 23462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:39.358042 23462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:40:39 dist-test-slave-z47j krb5kdc[23018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534439, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
I20260927 18:40:39.414170 23462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "39b3b328a4864022a2068f4017b8aebb"
format_stamp: "Formatted at 2026-09-27 18:40:39 on dist-test-slave-z47j"
server_key: "966ed0d1ff4382a4de346376cf965c67"
server_key_iv: "d04e1952ba3d674cd8a91a3ee57620bb"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:39.414696 23462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "39b3b328a4864022a2068f4017b8aebb"
format_stamp: "Formatted at 2026-09-27 18:40:39 on dist-test-slave-z47j"
server_key: "966ed0d1ff4382a4de346376cf965c67"
server_key_iv: "d04e1952ba3d674cd8a91a3ee57620bb"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:39.416558 23462 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:40:39.436626 23477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:39.437249 23462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 18:40:39.437408 23462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "39b3b328a4864022a2068f4017b8aebb"
format_stamp: "Formatted at 2026-09-27 18:40:39 on dist-test-slave-z47j"
server_key: "966ed0d1ff4382a4de346376cf965c67"
server_key_iv: "d04e1952ba3d674cd8a91a3ee57620bb"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:39.437563 23462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:39.457096 23462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:39.459424 23462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:39.459604 23462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:39.465303 23462 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:38531
I20260927 18:40:39.465329 23529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:38531 every 8 connection(s)
I20260927 18:40:39.465813 23462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:40:39.466104 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 23462
I20260927 18:40:39.466229 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:40:39.466445 12271 external_mini_cluster.cc:1428] Setting key bc44fafbd569a88ef41e495ce5bc764d
I20260927 18:40:39.466730 23530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:39.470599 23530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb: Bootstrap starting.
I20260927 18:40:39.471391 23530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:39.471714 23530 log.cc:824] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:39.472577 23530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb: No bootstrap required, opened a new log
Sep 27 18:40:39 dist-test-slave-z47j krb5kdc[23018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:40:39.475960 23530 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39b3b328a4864022a2068f4017b8aebb" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 38531 } }
I20260927 18:40:39.476152 23530 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:39.476194 23530 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39b3b328a4864022a2068f4017b8aebb, State: Initialized, Role: FOLLOWER
I20260927 18:40:39.476352 23530 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39b3b328a4864022a2068f4017b8aebb" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 38531 } }
I20260927 18:40:39.476444 23530 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:40:39.476475 23530 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:40:39.476516 23530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:39.477263 23530 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39b3b328a4864022a2068f4017b8aebb" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 38531 } }
I20260927 18:40:39.477372 23530 leader_election.cc:302] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39b3b328a4864022a2068f4017b8aebb; no voters:
I20260927 18:40:39.477545 23530 leader_election.cc:288] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:40:39.477694 23535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:40:39.477833 23530 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:40:39.477862 23535 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb [term 1 LEADER]: Becoming Leader. State: Replica: 39b3b328a4864022a2068f4017b8aebb, State: Running, Role: LEADER
I20260927 18:40:39.478250 23535 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39b3b328a4864022a2068f4017b8aebb" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 38531 } }
I20260927 18:40:39.479458 23535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39b3b328a4864022a2068f4017b8aebb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39b3b328a4864022a2068f4017b8aebb" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 38531 } } }
I20260927 18:40:39.479506 23536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39b3b328a4864022a2068f4017b8aebb. Latest consensus state: current_term: 1 leader_uuid: "39b3b328a4864022a2068f4017b8aebb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39b3b328a4864022a2068f4017b8aebb" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 38531 } } }
I20260927 18:40:39.479550 23535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb [sys.catalog]: This master's current role is: LEADER
I20260927 18:40:39.479575 23536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb [sys.catalog]: This master's current role is: LEADER
I20260927 18:40:39.479785 23549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:40:39.479813 23533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:39.467208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56720 (local address 127.11.251.254:38531)
0927 18:40:39.467400 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:39.467410 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:39.467442 (+ 32us) server_negotiation.cc:407] Connection header received
0927 18:40:39.468004 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:39.468011 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:39.468061 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:39.468233 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:39.468903 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:39.469975 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:39.471262 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:39.471972 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:39.472515 (+ 543us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:39.475136 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:39.475142 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:39.475142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:39.475154 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:39.477199 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:39.477905 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:39.477909 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:39.477909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:39.478047 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:39.478306 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:39.478307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:39.478308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:39.478693 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:39.478892 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:39.479371 (+ 479us) server_negotiation.cc:299] Negotiation successful
0927 18:40:39.479516 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":106,"threads_started":1}
I20260927 18:40:39.479912 23530 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260927 18:40:39.484058 23549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:40:39.487448 23549 catalog_manager.cc:1347] Generated new cluster ID: c0539bb58237410ea29d32adc1e9ab32
I20260927 18:40:39.487511 23549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:40:39.504221 23549 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:40:39.505225 23549 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:40:39.514086 23549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb: Generated new TSK 0
I20260927 18:40:39.514717 23549 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.251.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
I20260927 18:40:40.305943 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.251.212:33271/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:38531
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:37173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:40:40.427867 23575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:40.428143 23575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:40.428185 23575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:40.430707 23575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:40.430820 23575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:40.430884 23575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:40:40.433486 23575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:37173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.251.212:33271/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:38531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:40:40.433846 23575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:40.434326 23575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:40.441614 23583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:40.441679 23580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:40.441618 23581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:40.450747 23575 server_base.cc:1034] running on GCE node
I20260927 18:40:40.451148 23575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:40.451467 23575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:40.452656 23575 hybrid_clock.cc:648] HybridClock initialized: now 1790534440452627 us; error 45 us; skew 500 ppm
Sep 27 18:40:40 dist-test-slave-z47j krb5kdc[23018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534440, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:40.456318 23575 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:40:40.459631 23575 webserver.cc:460] Webserver started at http://127.11.251.193:40751/ using document root <none> and password file <none>
I20260927 18:40:40.459962 23575 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:40.460031 23575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:40.460189 23575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:40:40 dist-test-slave-z47j krb5kdc[23018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534440, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
I20260927 18:40:40.483810 23575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "ad188487dc9549f2a634d8432c54ee96"
format_stamp: "Formatted at 2026-09-27 18:40:40 on dist-test-slave-z47j"
server_key: "d3e5afa6707306d67a8eb891812262e6"
server_key_iv: "e7c8e6c2430005684e43f21f89934d2d"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:40.484336 23575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "ad188487dc9549f2a634d8432c54ee96"
format_stamp: "Formatted at 2026-09-27 18:40:40 on dist-test-slave-z47j"
server_key: "d3e5afa6707306d67a8eb891812262e6"
server_key_iv: "e7c8e6c2430005684e43f21f89934d2d"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:40.497633 23575 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.004s sys 0.001s
I20260927 18:40:40.515508 23592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:40.515722 23575 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:40:40.515821 23575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "ad188487dc9549f2a634d8432c54ee96"
format_stamp: "Formatted at 2026-09-27 18:40:40 on dist-test-slave-z47j"
server_key: "d3e5afa6707306d67a8eb891812262e6"
server_key_iv: "e7c8e6c2430005684e43f21f89934d2d"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:40.515959 23575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:40.548329 23575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:40.551035 23575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:40.551227 23575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:40.551561 23575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:40.552070 23575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:40:40.552124 23575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:40.552166 23575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:40:40.552184 23575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:40.563648 23575 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:35177
I20260927 18:40:40.563733 23707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:35177 every 8 connection(s)
I20260927 18:40:40.564215 23575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:40:40.564920 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 23575
I20260927 18:40:40.565053 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:40:40.565277 12271 external_mini_cluster.cc:1428] Setting key f9cf858c5a592cfc50a492bbab0848cc
WARNING: no policy specified for kudu/127.11.251.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.194@KRBTEST.COM" created.
Sep 27 18:40:40 dist-test-slave-z47j krb5kdc[23018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534440, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:40:40.592651 23710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:40.565842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:38531 (local address 127.11.251.193:50815)
0927 18:40:40.566088 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:40.566145 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:40.566834 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:40.567169 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:40.567174 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:40.567444 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:40.567751 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:40.567757 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:40.568894 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:40.568895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:40.569956 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:40.569959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:40.582980 (+ 13021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:40.582982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:40.583234 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:40.584006 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:40.584018 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:40.585996 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:40.590290 (+ 4294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:40.590306 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:40.590311 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:40.590839 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:40.591140 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:40.591141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:40.591141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:40.591325 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:40.591711 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:40.591720 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:40.591958 (+ 238us) client_negotiation.cc:769] Sending connection context
0927 18:40:40.592081 (+ 123us) client_negotiation.cc:240] Negotiation successful
0927 18:40:40.592314 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":120,"threads_started":1}
I20260927 18:40:40.593155 23711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:40.566122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:50815 (local address 127.11.251.254:38531)
0927 18:40:40.566297 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:40.566299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:40.566310 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:40:40.566969 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:40.566971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:40.567002 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:40.567088 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:40.567879 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:40.568793 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:40.582399 (+ 13606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:40.582766 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:40.583571 (+ 805us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:40.586148 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:40.586150 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:40.586151 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:40.586161 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:40.590139 (+ 3978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:40.590959 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:40.590960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:40.590960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:40.591036 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:40.591430 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:40.591431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:40.591431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:40.591621 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:40.592863 (+ 1242us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:40.592915 (+ 52us) server_negotiation.cc:299] Negotiation successful
0927 18:40:40.593045 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
I20260927 18:40:40.593720 23708 heartbeater.cc:347] Connected to a master server at 127.11.251.254:38531
I20260927 18:40:40.593847 23708 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:40.594110 23708 heartbeater.cc:511] Master 127.11.251.254:38531 requested a full tablet report, sending...
I20260927 18:40:40.594787 23494 ts_manager.cc:194] Registered new tserver with Master: ad188487dc9549f2a634d8432c54ee96 (127.11.251.193:35177)
I20260927 18:40:40.595773 23494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:50815
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
I20260927 18:40:40.604852 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.251.212:33271/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:38531
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:37173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:40:40 dist-test-slave-z47j krb5kdc[23018](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790534440, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260927 18:40:40.707593 23714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:40.707856 23714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:40.707893 23714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:40.709182 23714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:40.709244 23714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:40.709288 23714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:40:40.710996 23714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:37173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.251.212:33271/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:38531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:40:40.711297 23714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:40.711547 23714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:40.717818 23719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:40.717813 23722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:40.717825 23720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:40.717928 23714 server_base.cc:1034] running on GCE node
I20260927 18:40:40.718295 23714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:40.718605 23714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:40.719822 23714 hybrid_clock.cc:648] HybridClock initialized: now 1790534440719795 us; error 43 us; skew 500 ppm
Sep 27 18:40:40 dist-test-slave-z47j krb5kdc[23018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534440, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:40.729511 23714 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:40:40.730150 23714 webserver.cc:460] Webserver started at http://127.11.251.194:40841/ using document root <none> and password file <none>
I20260927 18:40:40.730572 23714 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:40.730650 23714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:40.730815 23714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:40:40 dist-test-slave-z47j krb5kdc[23018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534440, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
I20260927 18:40:40.744555 23714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "fc67765f2ef54144b998c5530948a4aa"
format_stamp: "Formatted at 2026-09-27 18:40:40 on dist-test-slave-z47j"
server_key: "42697bab41b15e9f5d1d01f310fb9dcb"
server_key_iv: "9a05ad8aa3fb93efd1e3cb932ad33ff7"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:40.745031 23714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "fc67765f2ef54144b998c5530948a4aa"
format_stamp: "Formatted at 2026-09-27 18:40:40 on dist-test-slave-z47j"
server_key: "42697bab41b15e9f5d1d01f310fb9dcb"
server_key_iv: "9a05ad8aa3fb93efd1e3cb932ad33ff7"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:40.746779 23714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:40:40.770390 23730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:40.770619 23714 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:40:40.770735 23714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "fc67765f2ef54144b998c5530948a4aa"
format_stamp: "Formatted at 2026-09-27 18:40:40 on dist-test-slave-z47j"
server_key: "42697bab41b15e9f5d1d01f310fb9dcb"
server_key_iv: "9a05ad8aa3fb93efd1e3cb932ad33ff7"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:40.770872 23714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:40.784374 23714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:40.788175 23714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:40.788389 23714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:40.788702 23714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:40.789170 23714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:40:40.789214 23714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:40.789252 23714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:40:40.789274 23714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:40.798826 23714 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:33579
I20260927 18:40:40.799309 23714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:40:40.801537 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 23714
I20260927 18:40:40.801702 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:40:40.802003 12271 external_mini_cluster.cc:1428] Setting key 684351816b9b74b577372bd93ad1b7e1
I20260927 18:40:40.803216 23843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:33579 every 8 connection(s)
Sep 27 18:40:40 dist-test-slave-z47j krb5kdc[23018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534440, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:40:40.815063 23846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:40.802755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:38531 (local address 127.11.251.194:57581)
0927 18:40:40.803609 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:40.803655 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:40.804365 (+ 710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:40.804773 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:40.804778 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:40.805064 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:40.805374 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:40.805379 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:40.806512 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:40.806513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:40.807817 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:40.807819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:40.808394 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:40.808395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:40.808583 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:40.809253 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:40.809266 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:40.811225 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:40.813188 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:40.813194 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:40.813197 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:40.813533 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:40.813786 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:40.813787 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:40.813787 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:40.813936 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:40.814273 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:40.814279 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:40.814474 (+ 195us) client_negotiation.cc:769] Sending connection context
0927 18:40:40.814600 (+ 126us) client_negotiation.cc:240] Negotiation successful
0927 18:40:40.814783 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"thread_start_us":86,"threads_started":1}
I20260927 18:40:40.815747 23711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:40.803055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:57581 (local address 127.11.251.254:38531)
0927 18:40:40.803113 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:40.803115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:40.803744 (+ 629us) server_negotiation.cc:407] Connection header received
0927 18:40:40.804505 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:40.804507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:40.804525 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:40.804558 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:40.805511 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:40.806378 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:40.807920 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:40.808225 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:40.808312 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:40.811339 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:40.811341 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:40.811341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:40.811350 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:40.813090 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:40.813647 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:40.813648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:40.813648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:40.813714 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:40.814017 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:40.814018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:40.814018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:40.814175 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:40.814244 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:40.815544 (+ 1300us) server_negotiation.cc:299] Negotiation successful
0927 18:40:40.815643 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 18:40:40.816717 23844 heartbeater.cc:347] Connected to a master server at 127.11.251.254:38531
I20260927 18:40:40.816836 23844 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:40.817061 23844 heartbeater.cc:511] Master 127.11.251.254:38531 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.251.195@KRBTEST.COM; defaulting to no policy
I20260927 18:40:40.817538 23494 ts_manager.cc:194] Registered new tserver with Master: fc67765f2ef54144b998c5530948a4aa (127.11.251.194:33579)
I20260927 18:40:40.818063 23494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:57581
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
I20260927 18:40:40.840961 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.251.212:33271/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:38531
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:37173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:40:40.978648 23849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:40.978991 23849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:40.979041 23849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:40.980907 23849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:40.980971 23849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:40.981027 23849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:40:40.983290 23849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:37173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.251.212:33271/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:38531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:40:40.983579 23849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:40.983888 23849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:40.989948 23855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:40.990275 23849 server_base.cc:1034] running on GCE node
W20260927 18:40:40.989990 23854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:40.990943 23857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:40.991196 23849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:40.991477 23849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:40.992645 23849 hybrid_clock.cc:648] HybridClock initialized: now 1790534440992631 us; error 35 us; skew 500 ppm
Sep 27 18:40:40 dist-test-slave-z47j krb5kdc[23018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534440, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:40.996713 23849 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:40:40.997427 23849 webserver.cc:460] Webserver started at http://127.11.251.195:44533/ using document root <none> and password file <none>
I20260927 18:40:40.997771 23849 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:40.997848 23849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:40.998003 23849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:40:40 dist-test-slave-z47j krb5kdc[23018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534440, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
I20260927 18:40:41.018885 23849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "a4865b4bd9d445ce8da50cb3806ec85e"
format_stamp: "Formatted at 2026-09-27 18:40:41 on dist-test-slave-z47j"
server_key: "e2cd9e513b88298e60b59e4a44d102dd"
server_key_iv: "0b9a404706cd54be3f829d212f21b8de"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:41.019492 23849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "a4865b4bd9d445ce8da50cb3806ec85e"
format_stamp: "Formatted at 2026-09-27 18:40:41 on dist-test-slave-z47j"
server_key: "e2cd9e513b88298e60b59e4a44d102dd"
server_key_iv: "0b9a404706cd54be3f829d212f21b8de"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:41.022451 23849 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:40:41.045480 23864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:41.045738 23849 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:40:41.045843 23849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "a4865b4bd9d445ce8da50cb3806ec85e"
format_stamp: "Formatted at 2026-09-27 18:40:41 on dist-test-slave-z47j"
server_key: "e2cd9e513b88298e60b59e4a44d102dd"
server_key_iv: "0b9a404706cd54be3f829d212f21b8de"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:41.045966 23849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:41.074579 23849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:41.077503 23849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:41.077759 23849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:41.078164 23849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:41.078609 23849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:40:41.078660 23849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:41.078696 23849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:40:41.078720 23849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:41.088406 23849 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:42459
I20260927 18:40:41.089148 23849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:40:41.089362 23977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:42459 every 8 connection(s)
I20260927 18:40:41.094936 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 23849
I20260927 18:40:41.095072 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:40:41.095323 12271 external_mini_cluster.cc:1428] Setting key c8e7b47b11a203a44a9fb4606efb28f7
Sep 27 18:40:41 dist-test-slave-z47j krb5kdc[23018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534440, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:40:41.103509 23711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:41.091028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:59319 (local address 127.11.251.254:38531)
0927 18:40:41.091087 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:41.091090 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:41.091222 (+ 132us) server_negotiation.cc:407] Connection header received
0927 18:40:41.092062 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:41.092064 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:41.092079 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:41.092153 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:41.093182 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.094069 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.095889 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.096228 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.096282 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:41.099508 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:41.099510 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:41.099511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:41.099521 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:41.101533 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:41.102155 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:41.102157 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:41.102158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:41.102223 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:41.102568 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:41.102569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:41.102570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:41.102736 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:41.102786 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:41.103283 (+ 497us) server_negotiation.cc:299] Negotiation successful
0927 18:40:41.103393 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 18:40:41.104301 23980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:41.090666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:38531 (local address 127.11.251.195:59319)
0927 18:40:41.090899 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:41.090956 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:41.091917 (+ 961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:41.092236 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:41.092241 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:41.092538 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:41.093052 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:41.093058 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.094194 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.094195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:41.095583 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:41.095586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.096476 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.096477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:41.096640 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:41.097330 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:41.097342 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:41.099371 (+ 2029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:41.101718 (+ 2347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:41.101727 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:41.101731 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:41.102060 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:41.102299 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:41.102299 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:41.102300 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:41.102461 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:41.102837 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:41.102845 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:41.103101 (+ 256us) client_negotiation.cc:769] Sending connection context
0927 18:40:41.103812 (+ 711us) client_negotiation.cc:240] Negotiation successful
0927 18:40:41.104018 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":105,"threads_started":1}
I20260927 18:40:41.104986 23978 heartbeater.cc:347] Connected to a master server at 127.11.251.254:38531
I20260927 18:40:41.105144 23978 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:41.105405 23978 heartbeater.cc:511] Master 127.11.251.254:38531 requested a full tablet report, sending...
I20260927 18:40:41.105973 23494 ts_manager.cc:194] Registered new tserver with Master: a4865b4bd9d445ce8da50cb3806ec85e (127.11.251.195:42459)
I20260927 18:40:41.106534 23494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:59319
I20260927 18:40:41.107669 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:40:41.119905 23711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:41.110097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56722 (local address 127.11.251.254:38531)
0927 18:40:41.110147 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:41.110149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:41.110404 (+ 255us) server_negotiation.cc:407] Connection header received
0927 18:40:41.110511 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:41.110512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:41.110524 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:41.110569 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:41.111507 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.112305 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.113439 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.113794 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.113841 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:41.115707 (+ 1866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:41.115709 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:41.115710 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:41.115718 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:41.118385 (+ 2667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:41.118890 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:41.118891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:41.118892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:41.118959 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:41.119199 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:41.119200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:41.119200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:41.119353 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:41.119446 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:41.119683 (+ 237us) server_negotiation.cc:299] Negotiation successful
0927 18:40:41.119775 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:40:41.128142 23494 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56722:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:40:41.132231 23494 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:40:41.141753 23992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:41.135673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:42459 (local address 127.0.0.1:36876)
0927 18:40:41.136737 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:41.136782 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:41.136865 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:41.137400 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:41.137401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:41.137404 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:41.137460 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:41.137462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.138867 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.138868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:41.140284 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:41.140287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.141369 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.141370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:41.141460 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:41.141478 (+ 18us) client_negotiation.cc:769] Sending connection context
0927 18:40:41.141527 (+ 49us) client_negotiation.cc:240] Negotiation successful
0927 18:40:41.141620 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1032,"mutex_wait_us":4,"thread_start_us":95,"threads_started":1}
I20260927 18:40:41.142275 23994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:41.136153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43632 (local address 127.11.251.194:33579)
0927 18:40:41.136669 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:41.136673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:41.137068 (+ 395us) server_negotiation.cc:407] Connection header received
0927 18:40:41.137132 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:41.137134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:41.137153 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:41.137246 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:41.137614 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.139070 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.140412 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.141497 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.141640 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:41.141708 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:41.142131 (+ 423us) server_negotiation.cc:299] Negotiation successful
0927 18:40:41.142183 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":95,"threads_started":1}
I20260927 18:40:41.142858 23993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:41.136144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36876 (local address 127.11.251.195:42459)
0927 18:40:41.137165 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:41.137169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:41.137180 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:40:41.137223 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:41.137225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:41.137241 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:41.137314 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:41.137552 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.138762 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.140425 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.141269 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.142565 (+ 1296us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:41.142685 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:41.142714 (+ 29us) server_negotiation.cc:299] Negotiation successful
0927 18:40:41.142762 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":978,"thread_start_us":97,"threads_started":1}
I20260927 18:40:41.143652 23912 tablet_service.cc:1467] Processing CreateTablet for tablet 7d159de47bb2422ba1d7dde863ae7601 (DEFAULT_TABLE table=test-table [id=6d6487734a474a8c9324bbd4c97b4ea1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:40:41.144136 23912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d159de47bb2422ba1d7dde863ae7601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:41.147923 23995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:41.136355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49222 (local address 127.11.251.193:35177)
0927 18:40:41.136583 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:41.136589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:41.141750 (+ 5161us) server_negotiation.cc:407] Connection header received
0927 18:40:41.141886 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:41.141890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:41.141915 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:41.141982 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:41.142227 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.143332 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.146546 (+ 3214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.147382 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.147442 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:41.147506 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:41.147760 (+ 254us) server_negotiation.cc:299] Negotiation successful
0927 18:40:41.147822 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":72,"threads_started":1}
I20260927 18:40:41.142060 23990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:41.135673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:33579 (local address 127.0.0.1:43632)
0927 18:40:41.136972 (+ 1299us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:41.136987 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:41.137038 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:41.137363 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:41.137367 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:41.137370 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:41.137458 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:41.137461 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.139175 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.139176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:41.140287 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:41.140290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.141746 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.141747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:41.141811 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:41.141816 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:40:41.141933 (+ 117us) client_negotiation.cc:240] Negotiation successful
0927 18:40:41.141982 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1280,"mutex_wait_us":10,"thread_start_us":112,"threads_started":1}
I20260927 18:40:41.148494 23991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:41.135912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:35177 (local address 127.0.0.1:49222)
0927 18:40:41.141654 (+ 5742us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:41.141666 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:41.141785 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:41.142106 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:41.142107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:41.142109 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:41.142151 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:41.142153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.143442 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.143443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:41.144524 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:41.144527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.147581 (+ 3054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.147582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:41.147650 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:41.147655 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:40:41.148332 (+ 677us) client_negotiation.cc:240] Negotiation successful
0927 18:40:41.148411 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5720,"thread_start_us":65,"threads_started":1}
I20260927 18:40:41.149712 23640 tablet_service.cc:1467] Processing CreateTablet for tablet 7d159de47bb2422ba1d7dde863ae7601 (DEFAULT_TABLE table=test-table [id=6d6487734a474a8c9324bbd4c97b4ea1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:40:41.149698 23772 tablet_service.cc:1467] Processing CreateTablet for tablet 7d159de47bb2422ba1d7dde863ae7601 (DEFAULT_TABLE table=test-table [id=6d6487734a474a8c9324bbd4c97b4ea1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:40:41.150085 23640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d159de47bb2422ba1d7dde863ae7601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:41.150097 23772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d159de47bb2422ba1d7dde863ae7601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:41.153322 23997 tablet_bootstrap.cc:492] T 7d159de47bb2422ba1d7dde863ae7601 P ad188487dc9549f2a634d8432c54ee96: Bootstrap starting.
I20260927 18:40:41.153987 23996 tablet_bootstrap.cc:492] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e: Bootstrap starting.
I20260927 18:40:41.155112 23996 tablet_bootstrap.cc:654] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:41.155112 23997 tablet_bootstrap.cc:654] T 7d159de47bb2422ba1d7dde863ae7601 P ad188487dc9549f2a634d8432c54ee96: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:41.155498 23997 log.cc:824] T 7d159de47bb2422ba1d7dde863ae7601 P ad188487dc9549f2a634d8432c54ee96: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:41.155512 23996 log.cc:824] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:41.155938 23998 tablet_bootstrap.cc:492] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa: Bootstrap starting.
I20260927 18:40:41.156919 23998 tablet_bootstrap.cc:654] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:41.157238 23998 log.cc:824] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:41.158126 23996 tablet_bootstrap.cc:492] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e: No bootstrap required, opened a new log
I20260927 18:40:41.158209 23996 ts_tablet_manager.cc:1397] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260927 18:40:41.160305 23996 raft_consensus.cc:348] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc67765f2ef54144b998c5530948a4aa" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 33579 } } peers { permanent_uuid: "a4865b4bd9d445ce8da50cb3806ec85e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 42459 } } peers { permanent_uuid: "ad188487dc9549f2a634d8432c54ee96" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 35177 } }
I20260927 18:40:41.160506 23996 raft_consensus.cc:374] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:41.160557 23996 raft_consensus.cc:729] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4865b4bd9d445ce8da50cb3806ec85e, State: Initialized, Role: FOLLOWER
I20260927 18:40:41.160717 23996 consensus_queue.cc:260] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc67765f2ef54144b998c5530948a4aa" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 33579 } } peers { permanent_uuid: "a4865b4bd9d445ce8da50cb3806ec85e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 42459 } } peers { permanent_uuid: "ad188487dc9549f2a634d8432c54ee96" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 35177 } }
I20260927 18:40:41.161031 23996 ts_tablet_manager.cc:1428] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:40:41.161199 23978 heartbeater.cc:503] Master 127.11.251.254:38531 was elected leader, sending a full tablet report...
I20260927 18:40:41.161620 23998 tablet_bootstrap.cc:492] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa: No bootstrap required, opened a new log
I20260927 18:40:41.161690 23998 ts_tablet_manager.cc:1397] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260927 18:40:41.163782 23998 raft_consensus.cc:348] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc67765f2ef54144b998c5530948a4aa" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 33579 } } peers { permanent_uuid: "a4865b4bd9d445ce8da50cb3806ec85e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 42459 } } peers { permanent_uuid: "ad188487dc9549f2a634d8432c54ee96" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 35177 } }
I20260927 18:40:41.163978 23998 raft_consensus.cc:374] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:41.164011 23998 raft_consensus.cc:729] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc67765f2ef54144b998c5530948a4aa, State: Initialized, Role: FOLLOWER
I20260927 18:40:41.164189 23998 consensus_queue.cc:260] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc67765f2ef54144b998c5530948a4aa" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 33579 } } peers { permanent_uuid: "a4865b4bd9d445ce8da50cb3806ec85e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 42459 } } peers { permanent_uuid: "ad188487dc9549f2a634d8432c54ee96" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 35177 } }
I20260927 18:40:41.164492 23998 ts_tablet_manager.cc:1428] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:40:41.164551 23844 heartbeater.cc:503] Master 127.11.251.254:38531 was elected leader, sending a full tablet report...
I20260927 18:40:41.165380 23997 tablet_bootstrap.cc:492] T 7d159de47bb2422ba1d7dde863ae7601 P ad188487dc9549f2a634d8432c54ee96: No bootstrap required, opened a new log
I20260927 18:40:41.165462 23997 ts_tablet_manager.cc:1397] T 7d159de47bb2422ba1d7dde863ae7601 P ad188487dc9549f2a634d8432c54ee96: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260927 18:40:41.168190 23997 raft_consensus.cc:348] T 7d159de47bb2422ba1d7dde863ae7601 P ad188487dc9549f2a634d8432c54ee96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc67765f2ef54144b998c5530948a4aa" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 33579 } } peers { permanent_uuid: "a4865b4bd9d445ce8da50cb3806ec85e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 42459 } } peers { permanent_uuid: "ad188487dc9549f2a634d8432c54ee96" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 35177 } }
I20260927 18:40:41.168383 23997 raft_consensus.cc:374] T 7d159de47bb2422ba1d7dde863ae7601 P ad188487dc9549f2a634d8432c54ee96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:41.168419 23997 raft_consensus.cc:729] T 7d159de47bb2422ba1d7dde863ae7601 P ad188487dc9549f2a634d8432c54ee96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad188487dc9549f2a634d8432c54ee96, State: Initialized, Role: FOLLOWER
I20260927 18:40:41.168581 23997 consensus_queue.cc:260] T 7d159de47bb2422ba1d7dde863ae7601 P ad188487dc9549f2a634d8432c54ee96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc67765f2ef54144b998c5530948a4aa" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 33579 } } peers { permanent_uuid: "a4865b4bd9d445ce8da50cb3806ec85e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 42459 } } peers { permanent_uuid: "ad188487dc9549f2a634d8432c54ee96" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 35177 } }
I20260927 18:40:41.168902 23708 heartbeater.cc:503] Master 127.11.251.254:38531 was elected leader, sending a full tablet report...
I20260927 18:40:41.169132 23997 ts_tablet_manager.cc:1428] T 7d159de47bb2422ba1d7dde863ae7601 P ad188487dc9549f2a634d8432c54ee96: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
Sep 27 18:40:41 dist-test-slave-z47j krb5kdc[23018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534440, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
W20260927 18:40:41.302577 23845 tablet.cc:2369] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:40:41.315418 23709 tablet.cc:2369] T 7d159de47bb2422ba1d7dde863ae7601 P ad188487dc9549f2a634d8432c54ee96: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:40:41.341972 23979 tablet.cc:2369] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:40:41.436880 24003 raft_consensus.cc:482] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:40:41.437072 24003 raft_consensus.cc:504] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc67765f2ef54144b998c5530948a4aa" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 33579 } } peers { permanent_uuid: "a4865b4bd9d445ce8da50cb3806ec85e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 42459 } } peers { permanent_uuid: "ad188487dc9549f2a634d8432c54ee96" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 35177 } }
I20260927 18:40:41.437551 24003 leader_election.cc:288] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4865b4bd9d445ce8da50cb3806ec85e (127.11.251.195:42459), ad188487dc9549f2a634d8432c54ee96 (127.11.251.193:35177)
I20260927 18:40:41.442273 23993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:41.438419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:41597 (local address 127.11.251.195:42459)
0927 18:40:41.438464 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:41.438466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:41.438599 (+ 133us) server_negotiation.cc:407] Connection header received
0927 18:40:41.438708 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:41.438709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:41.438720 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:41.438748 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:41.438984 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.439773 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.441268 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.441837 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.441886 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:41.441898 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:41.442124 (+ 226us) server_negotiation.cc:299] Negotiation successful
0927 18:40:41.442170 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 18:40:41.442646 24006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:41.437904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:42459 (local address 127.11.251.194:41597)
0927 18:40:41.438526 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:41.438538 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:41.438629 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:41.438824 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:41.438825 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:41.438828 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:41.438916 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:41.438919 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.439904 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.439905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:41.441159 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:41.441163 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.441957 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.441958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:41.442039 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:41.442044 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:40:41.442447 (+ 403us) client_negotiation.cc:240] Negotiation successful
0927 18:40:41.442508 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":103,"threads_started":1}
I20260927 18:40:41.443024 23932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d159de47bb2422ba1d7dde863ae7601" candidate_uuid: "fc67765f2ef54144b998c5530948a4aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4865b4bd9d445ce8da50cb3806ec85e" is_pre_election: true
I20260927 18:40:41.443233 23932 raft_consensus.cc:2454] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc67765f2ef54144b998c5530948a4aa in term 0.
I20260927 18:40:41.443516 23733 leader_election.cc:302] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4865b4bd9d445ce8da50cb3806ec85e, fc67765f2ef54144b998c5530948a4aa; no voters:
I20260927 18:40:41.443706 24003 raft_consensus.cc:2781] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:40:41.443763 24003 raft_consensus.cc:482] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:40:41.443790 24003 raft_consensus.cc:3037] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:41.444834 24003 raft_consensus.cc:504] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc67765f2ef54144b998c5530948a4aa" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 33579 } } peers { permanent_uuid: "a4865b4bd9d445ce8da50cb3806ec85e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 42459 } } peers { permanent_uuid: "ad188487dc9549f2a634d8432c54ee96" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 35177 } }
I20260927 18:40:41.444988 24003 leader_election.cc:288] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa [CANDIDATE]: Term 1 election: Requested vote from peers a4865b4bd9d445ce8da50cb3806ec85e (127.11.251.195:42459), ad188487dc9549f2a634d8432c54ee96 (127.11.251.193:35177)
I20260927 18:40:41.445211 23932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d159de47bb2422ba1d7dde863ae7601" candidate_uuid: "fc67765f2ef54144b998c5530948a4aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4865b4bd9d445ce8da50cb3806ec85e"
I20260927 18:40:41.445292 23932 raft_consensus.cc:3037] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:41.446250 23932 raft_consensus.cc:2454] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc67765f2ef54144b998c5530948a4aa in term 1.
I20260927 18:40:41.446425 23733 leader_election.cc:302] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4865b4bd9d445ce8da50cb3806ec85e, fc67765f2ef54144b998c5530948a4aa; no voters:
I20260927 18:40:41.446514 24003 raft_consensus.cc:2781] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:40:41.446688 24003 raft_consensus.cc:686] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa [term 1 LEADER]: Becoming Leader. State: Replica: fc67765f2ef54144b998c5530948a4aa, State: Running, Role: LEADER
I20260927 18:40:41.446818 24003 consensus_queue.cc:237] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc67765f2ef54144b998c5530948a4aa" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 33579 } } peers { permanent_uuid: "a4865b4bd9d445ce8da50cb3806ec85e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 42459 } } peers { permanent_uuid: "ad188487dc9549f2a634d8432c54ee96" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 35177 } }
I20260927 18:40:41.447796 23494 catalog_manager.cc:5495] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa reported cstate change: term changed from 0 to 1, leader changed from <none> to fc67765f2ef54144b998c5530948a4aa (127.11.251.194). New cstate: current_term: 1 leader_uuid: "fc67765f2ef54144b998c5530948a4aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc67765f2ef54144b998c5530948a4aa" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 33579 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4865b4bd9d445ce8da50cb3806ec85e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 42459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad188487dc9549f2a634d8432c54ee96" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 35177 } health_report { overall_health: UNKNOWN } } }
I20260927 18:40:41.456941 23995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:41.438296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:40885 (local address 127.11.251.193:35177)
0927 18:40:41.438358 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:41.438361 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:41.448609 (+ 10248us) server_negotiation.cc:407] Connection header received
0927 18:40:41.448723 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:41.448725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:41.448738 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:41.448768 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:41.452265 (+ 3497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.453050 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.455756 (+ 2706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.456468 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.456551 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:41.456567 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:41.456794 (+ 227us) server_negotiation.cc:299] Negotiation successful
0927 18:40:41.456851 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 18:40:41.457547 24006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:41.438119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:35177 (local address 127.11.251.194:40885)
0927 18:40:41.448536 (+ 10417us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:41.448550 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:41.448642 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:41.452076 (+ 3434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:41.452078 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:41.452081 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:41.452141 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:41.452143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.454519 (+ 2376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.454520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:41.455594 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:41.455597 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.456643 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.456644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:41.456711 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:41.456715 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:40:41.457392 (+ 677us) client_negotiation.cc:240] Negotiation successful
0927 18:40:41.457454 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10400,"thread_start_us":46,"threads_started":1}
I20260927 18:40:41.458135 23659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d159de47bb2422ba1d7dde863ae7601" candidate_uuid: "fc67765f2ef54144b998c5530948a4aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad188487dc9549f2a634d8432c54ee96" is_pre_election: true
I20260927 18:40:41.458348 23659 raft_consensus.cc:2454] T 7d159de47bb2422ba1d7dde863ae7601 P ad188487dc9549f2a634d8432c54ee96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc67765f2ef54144b998c5530948a4aa in term 0.
I20260927 18:40:41.458484 23657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d159de47bb2422ba1d7dde863ae7601" candidate_uuid: "fc67765f2ef54144b998c5530948a4aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad188487dc9549f2a634d8432c54ee96"
I20260927 18:40:41.458556 23657 raft_consensus.cc:3037] T 7d159de47bb2422ba1d7dde863ae7601 P ad188487dc9549f2a634d8432c54ee96 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:41.459628 23657 raft_consensus.cc:2454] T 7d159de47bb2422ba1d7dde863ae7601 P ad188487dc9549f2a634d8432c54ee96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc67765f2ef54144b998c5530948a4aa in term 1.
I20260927 18:40:41.498178 23994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:41.494535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43636 (local address 127.11.251.194:33579)
0927 18:40:41.494595 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:41.494597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:41.494608 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:40:41.494693 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:41.494694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:41.494721 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:41.494794 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:40:41.495035 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.495792 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.496883 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:41.497192 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:41.497252 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:41.497457 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:40:41.497667 (+ 210us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:40:41.497762 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:41.498055 (+ 293us) server_negotiation.cc:299] Negotiation successful
0927 18:40:41.498083 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 18:40:41.502317 23657 raft_consensus.cc:1261] T 7d159de47bb2422ba1d7dde863ae7601 P ad188487dc9549f2a634d8432c54ee96 [term 1 FOLLOWER]: Refusing update from remote peer fc67765f2ef54144b998c5530948a4aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:40:41.502939 24007 consensus_queue.cc:1035] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad188487dc9549f2a634d8432c54ee96" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 35177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:40:41.508428 23932 raft_consensus.cc:1261] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e [term 1 FOLLOWER]: Refusing update from remote peer fc67765f2ef54144b998c5530948a4aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:40:41.508985 24003 consensus_queue.cc:1035] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4865b4bd9d445ce8da50cb3806ec85e" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 42459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:40:41.511778 24010 mvcc.cc:205] Tried to move back new op lower bound from 7334029072389967872 to 7334029072166817792. Current Snapshot: MvccSnapshot[applied={T|T < 7334029072389967872}]
I20260927 18:40:41.512580 24011 mvcc.cc:205] Tried to move back new op lower bound from 7334029072389967872 to 7334029072166817792. Current Snapshot: MvccSnapshot[applied={T|T < 7334029072389967872}]
I20260927 18:40:41.514904 23494 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:40:41.515028 23494 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:40:41.516010 23494 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39b3b328a4864022a2068f4017b8aebb: Sending DeleteTablet for 3 replicas of tablet 7d159de47bb2422ba1d7dde863ae7601
I20260927 18:40:41.516559 23912 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d159de47bb2422ba1d7dde863ae7601 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:40:41 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:36876
I20260927 18:40:41.516849 23640 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d159de47bb2422ba1d7dde863ae7601 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:40:41 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:49222
I20260927 18:40:41.517278 23772 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d159de47bb2422ba1d7dde863ae7601 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:40:41 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:43632
I20260927 18:40:41.517693 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 23575
I20260927 18:40:41.522957 24021 tablet_replica.cc:330] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa: stopping tablet replica
I20260927 18:40:41.523176 24021 raft_consensus.cc:2229] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:40:41.523443 24021 raft_consensus.cc:2258] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:40:41.524133 24021 ts_tablet_manager.cc:1905] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:40:41.526304 24014 mvcc.cc:205] Tried to move back new op lower bound from 7334029072389967872 to 7334029072166817792. Current Snapshot: MvccSnapshot[applied={T|T < 7334029072389967872}]
I20260927 18:40:41.526520 24021 ts_tablet_manager.cc:1918] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:40:41.526608 24021 log.cc:1196] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/7d159de47bb2422ba1d7dde863ae7601
I20260927 18:40:41.526927 24021 ts_tablet_manager.cc:1939] T 7d159de47bb2422ba1d7dde863ae7601 P fc67765f2ef54144b998c5530948a4aa: Deleting consensus metadata
I20260927 18:40:41.526558 24019 tablet_replica.cc:330] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e: stopping tablet replica
I20260927 18:40:41.527210 24019 raft_consensus.cc:2229] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:40:41.527483 23479 catalog_manager.cc:4841] TS fc67765f2ef54144b998c5530948a4aa (127.11.251.194:33579): tablet 7d159de47bb2422ba1d7dde863ae7601 (table test-table [id=6d6487734a474a8c9324bbd4c97b4ea1]) successfully deleted
I20260927 18:40:41.528015 24019 raft_consensus.cc:2258] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:40:41.538777 24019 ts_tablet_manager.cc:1905] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:40:41.540891 24019 ts_tablet_manager.cc:1918] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:40:41.540994 24019 log.cc:1196] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/7d159de47bb2422ba1d7dde863ae7601
I20260927 18:40:41.541683 24019 ts_tablet_manager.cc:1939] T 7d159de47bb2422ba1d7dde863ae7601 P a4865b4bd9d445ce8da50cb3806ec85e: Deleting consensus metadata
I20260927 18:40:41.542263 23480 catalog_manager.cc:4841] TS a4865b4bd9d445ce8da50cb3806ec85e (127.11.251.195:42459): tablet 7d159de47bb2422ba1d7dde863ae7601 (table test-table [id=6d6487734a474a8c9324bbd4c97b4ea1]) successfully deleted
W20260927 18:40:41.551826 23480 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.251.193:35177 (error 108)
I20260927 18:40:41.553042 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 23714
I20260927 18:40:41.555217 23991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:41.555059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:35177 (local address 127.0.0.1:49236)
0927 18:40:41.555125 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:41.555167 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.193:35177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260927 18:40:41.555372 23480 catalog_manager.cc:4584] TS ad188487dc9549f2a634d8432c54ee96 (127.11.251.193:35177): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7d159de47bb2422ba1d7dde863ae7601: Network error: Client connection negotiation failed: client connection to 127.11.251.193:35177: connect: Connection refused (error 111)
I20260927 18:40:41.560417 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 23849
Sep 27 18:40:41 dist-test-slave-z47j krb5kdc[23018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534440, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
I20260927 18:40:41.567039 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 23462
I20260927 18:40:41.573549 12271 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260927 18:40:41.918828 12271 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260927 18:40:41.918991 12271 mini_ranger.cc:66] Stopping Ranger...
I20260927 18:40:42.274420 12271 mini_ranger.cc:68] Stopped Ranger
2026-09-27 18:40:42.274 UTC [23048] LOG: received smart shutdown request
2026-09-27 18:40:42.278 UTC [23048] LOG: background worker "logical replication launcher" (PID 23057) exited with exit code 1
2026-09-27 18:40:42.279 UTC [23052] LOG: shutting down
2026-09-27 18:40:42.483 UTC [23048] LOG: database system is shut down
2026-09-27T18:40:42Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51415 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:40:42 dist-test-slave-z47j krb5kdc[24033](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:40:42 dist-test-slave-z47j krb5kdc[24033](info): set up 2 sockets
Sep 27 18:40:42 dist-test-slave-z47j krb5kdc[24033](info): commencing operation
krb5kdc: starting...
W20260927 18:40:43.645994 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:40:43 dist-test-slave-z47j krb5kdc[24033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:40:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:40:43Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.251.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.251.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.251.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/rangeradmin_127.11.251.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.251.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/rangeradmin_127.11.251.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.251.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.251.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.251.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/rangerlookup_127.11.251.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.251.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/rangerlookup_127.11.251.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.251.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.251.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.251.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/HTTP_127.11.251.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.251.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/HTTP_127.11.251.212@KRBTEST.COM.keytab.
I20260927 18:40:43.829381 12271 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskbSvNM2/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/postgres -l logfile start
2026-09-27 18:40:46.536 UTC [24063] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-27 18:40:46.536 UTC [24063] LOG: listening on IPv4 address "127.11.251.212", port 39105
2026-09-27 18:40:46.541 UTC [24063] LOG: listening on Unix socket "/tmp/.s.PGSQL.39105"
2026-09-27 18:40:46.571 UTC [24066] LOG: database system was shut down at 2026-09-27 18:40:44 UTC
2026-09-27 18:40:46.574 UTC [24063] LOG: database system is ready to accept connections
I20260927 18:40:47.526239 12271 mini_postgres.cc:96] Postgres bound to 39105
2026-09-27 18:40:47.533 UTC [24074] FATAL: database "slave" does not exist
127.11.251.212:39105 - accepting connections
I20260927 18:40:47.533629 12271 mini_ranger.cc:161] Starting Ranger...
I20260927 18:40:47.549371 12271 mini_ranger.cc:84] Created miniranger Postgres user
I20260927 18:40:48.114415 12271 mini_ranger.cc:87] Created ranger Postgres database
I20260927 18:40:48.114507 12271 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ranger-admin
2026-09-27 18:40:48,410 [I] DB FLAVOR :POSTGRES
2026-09-27 18:40:48,410 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 18:40:48,410 [I] Checking connection..
2026-09-27 18:40:48,411 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-27 18:40:48,664 [I] Checking connection passed.
2026-09-27 18:40:48,665 [I] --------- Verifying version history table ---------
2026-09-27 18:40:48,665 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 18:40:48,957 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 18:40:48,958 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 18:40:49,239 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 18:40:49,240 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-27 18:40:49,240 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-27 18:40:49.492 UTC [24138] WARNING: there is no transaction in progress
2026-09-27 18:40:49,500 [I] create_dbversion_catalog.sql file imported successfully
2026-09-27 18:40:49,500 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 18:40:49,793 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-27 18:40:49,794 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 18:40:49,794 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-27 18:40:50,074 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-27 18:40:50,335 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-z47j.c.gcp-upstream.internal','N') ;"
2026-09-27 18:40:50,592 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 18:40:50,860 [I] Table x_portal_user does not exist in database ranger
2026-09-27 18:40:50,860 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 18:40:51,145 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-27 18:40:51,146 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 18:40:51,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 18:40:51,660 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-27 18:40:51,660 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-27 18:40:52.699 UTC [24246] WARNING: there is no transaction in progress
2026-09-27 18:40:52.718 UTC [24246] WARNING: there is no transaction in progress
2026-09-27 18:40:52.737 UTC [24246] WARNING: there is no transaction in progress
2026-09-27 18:40:52.753 UTC [24246] WARNING: there is no transaction in progress
2026-09-27 18:40:52.771 UTC [24246] WARNING: there is no transaction in progress
2026-09-27 18:40:52.791 UTC [24246] WARNING: there is no transaction in progress
2026-09-27 18:40:52.851 UTC [24246] WARNING: there is no transaction in progress
2026-09-27 18:40:52.858 UTC [24246] WARNING: there is no transaction in progress
2026-09-27 18:40:52.874 UTC [24246] WARNING: there is no transaction in progress
2026-09-27 18:40:52.886 UTC [24246] WARNING: there is no transaction in progress
2026-09-27 18:40:52.898 UTC [24246] WARNING: there is no transaction in progress
2026-09-27 18:40:52.915 UTC [24246] WARNING: there is no transaction in progress
2026-09-27 18:40:52.926 UTC [24246] WARNING: there is no transaction in progress
2026-09-27 18:40:52.933 UTC [24246] WARNING: there is no transaction in progress
2026-09-27 18:40:52.940 UTC [24246] WARNING: there is no transaction in progress
2026-09-27 18:40:52.948 UTC [24246] WARNING: there is no transaction in progress
2026-09-27 18:40:53,712 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-27 18:40:53,713 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-27 18:40:53,994 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-27 18:40:53,995 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 18:40:54,284 [I] Table x_portal_user already exists in database 'ranger'
2026-09-27 18:40:54,285 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 18:40:54,524 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-27 18:40:54,524 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 18:40:54,810 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 18:40:55,088 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-z47j.c.gcp-upstream.internal';"
2026-09-27 18:40:55,365 [I] CORE_DB_SCHEMA import status has been updated
2026-09-27 18:40:55,366 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-27 18:40:55,639 [I] DB_PATCHES have already been applied
I20260927 18:40:55.646070 12271 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ranger-admin:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskbSvNM2/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260927 18:40:55.646348 12271 mini_ranger.cc:193] Using host: 127.11.251.212
I20260927 18:40:55.650377 12271 mini_ranger.cc:227] Ranger admin URL: http://127.11.251.212:57067
Sep 27, 2026 6:40:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 6:40:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 6:40:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ranger-admin/ews/webapp
Sep 27, 2026 6:40:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 27, 2026 6:40:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 27, 2026 6:40:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ranger-admin/ews/webapp].
Sep 27, 2026 6:40:56 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260927 18:40:56.800642 12271 mini_ranger.cc:160] Time spent starting Ranger: real 9.267s user 0.001s sys 0.009s
Sep 27, 2026 6:41:18 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 27, 2026 6:41:18 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 27, 2026 6:41:18 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 27, 2026 6:41:18 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 27, 2026 6:41:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 27, 2026 6:41:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 27, 2026 6:41:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 27, 2026 6:41:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 27, 2026 6:41:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 27, 2026 6:41:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 27, 2026 6:41:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 27, 2026 6:41:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 27, 2026 6:41:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 27, 2026 6:41:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 27, 2026 6:41:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 27, 2026 6:41:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 27, 2026 6:41:18 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 27, 2026 6:41:19 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260927 18:41:20.615234 12271 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.251.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.251.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.251.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/rangerkms_127.11.251.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.251.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/rangerkms_127.11.251.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.251.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.251.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.251.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/HTTP_127.11.251.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.251.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/HTTP_127.11.251.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 27 18:41:20 dist-test-slave-z47j krb5kdc[24033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534480, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260927 18:41:20.710798 12271 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260927 18:41:20.721462 12271 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260927 18:41:21.745570 12271 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260927 18:41:21.745757 12271 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ranger-kms
I20260927 18:41:21.745779 12271 mini_ranger_kms.cc:227] Using postgres at 127.11.251.212:39105
2026-09-27 18:41:21,943 [I] DB FLAVOR :POSTGRES
2026-09-27 18:41:21,943 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 18:41:21,944 [I] Checking connection
2026-09-27 18:41:21,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-27 18:41:22,195 [I] connection success
2026-09-27 18:41:22,195 [I] --------- Verifying Ranger DB tables ---------
2026-09-27 18:41:22,195 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-27 18:41:22,195 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-27 18:41:22,503 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-27 18:41:22,503 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 18:41:22,503 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-27 18:41:22,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.251.212:39105/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-27 18:41:22,822 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260927 18:41:23.394178 12271 mini_ranger_kms.cc:310] Created kms service
I20260927 18:41:23.616554 12271 mini_ranger_kms.cc:326] Created kudu user
I20260927 18:41:23.668742 12271 mini_ranger_kms.cc:343] Created rangerkms user
Sep 27 18:41:23 dist-test-slave-z47j krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534480, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
I20260927 18:41:24.086663 12271 mini_ranger_kms.cc:384] Added ranger policy
I20260927 18:41:24.086928 12271 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ranger-kms:/tmp/dist-test-taskbSvNM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskbSvNM2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260927 18:41:24.086961 12271 mini_ranger_kms.cc:240] Using host: 127.11.251.212
I20260927 18:41:24.091835 12271 mini_ranger_kms.cc:276] Ranger KMS PID: 24427
I20260927 18:41:24.091971 12271 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.251.212:47113
Sep 27, 2026 6:41:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 6:41:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 6:41:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 27, 2026 6:41:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 27, 2026 6:41:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ranger-kms/ews/webapp].
Sep 27, 2026 6:41:25 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260927 18:41:26.414624 12271 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.704s user 0.016s sys 0.167s
I20260927 18:41:26.414762 12271 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260927 18:41:26.414819 12271 mini_ranger_kms.cc:401] 127.11.251.212:47113/kms/v1/keys
Sep 27 18:41:26 dist-test-slave-z47j krb5kdc[24033](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790534486, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.251.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 27 18:41:26 dist-test-slave-z47j krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534486, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.251.212@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
Sep 27 18:41:27 dist-test-slave-z47j krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534486, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.251.212@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
Sep 27 18:41:29 dist-test-slave-z47j krb5kdc[24033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:41:29.690223 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:39281
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:42727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:39281
--ranger_config_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.251.212:47113/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:29.781055 24477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:29.781307 24477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:29.781329 24477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:29.782670 24477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:29.782719 24477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:29.782734 24477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:29.782747 24477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:29.782759 24477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:29.784408 24477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:42727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.251.212:47113/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:39281
--ranger_config_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:39281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:29.784679 24477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:29.784968 24477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:29.791004 24485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:29.791034 24483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:29.791095 24477 server_base.cc:1034] running on GCE node
W20260927 18:41:29.791261 24482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:29.791601 24477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:29.791944 24477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:29.793128 24477 hybrid_clock.cc:648] HybridClock initialized: now 1790534489793103 us; error 44 us; skew 500 ppm
Sep 27 18:41:29 dist-test-slave-z47j krb5kdc[24033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534489, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:29.807786 24477 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:41:29.808543 24477 webserver.cc:460] Webserver started at http://127.11.251.254:40215/ using document root <none> and password file <none>
I20260927 18:41:29.808857 24477 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:29.808918 24477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:29.809072 24477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:41:29 dist-test-slave-z47j krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534489, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
I20260927 18:41:29.865653 24477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "7e6fb6bd973f4c019e9453cdafd71f37"
format_stamp: "Formatted at 2026-09-27 18:41:29 on dist-test-slave-z47j"
server_key: "efb0899c721274779025cff8cf409e73"
server_key_iv: "71de4fab8f2382dd55f48a6c2844545e"
server_key_version: "kuduclusterkey@0"
I20260927 18:41:29.866173 24477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "7e6fb6bd973f4c019e9453cdafd71f37"
format_stamp: "Formatted at 2026-09-27 18:41:29 on dist-test-slave-z47j"
server_key: "efb0899c721274779025cff8cf409e73"
server_key_iv: "71de4fab8f2382dd55f48a6c2844545e"
server_key_version: "kuduclusterkey@0"
I20260927 18:41:29.868225 24477 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260927 18:41:29.886029 24492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:29.886278 24477 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:41:29.886483 24477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "7e6fb6bd973f4c019e9453cdafd71f37"
format_stamp: "Formatted at 2026-09-27 18:41:29 on dist-test-slave-z47j"
server_key: "efb0899c721274779025cff8cf409e73"
server_key_iv: "71de4fab8f2382dd55f48a6c2844545e"
server_key_version: "kuduclusterkey@0"
I20260927 18:41:29.886631 24477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:29.899080 24477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:29.901546 24477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:29.901789 24477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:29.907415 24477 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:39281
I20260927 18:41:29.907509 24544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:39281 every 8 connection(s)
I20260927 18:41:29.907909 24477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:29.908710 24545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:29.911491 24545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37: Bootstrap starting.
I20260927 18:41:29.912191 24545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:29.912514 24545 log.cc:824] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:29.913230 24545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37: No bootstrap required, opened a new log
I20260927 18:41:29.915506 24545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e6fb6bd973f4c019e9453cdafd71f37" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 39281 } }
I20260927 18:41:29.915668 24545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:29.915691 24545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6fb6bd973f4c019e9453cdafd71f37, State: Initialized, Role: FOLLOWER
I20260927 18:41:29.915855 24545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e6fb6bd973f4c019e9453cdafd71f37" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 39281 } }
I20260927 18:41:29.915941 24545 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:29.915966 24545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:29.916034 24545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:29.916632 24545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e6fb6bd973f4c019e9453cdafd71f37" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 39281 } }
I20260927 18:41:29.916723 24545 leader_election.cc:302] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e6fb6bd973f4c019e9453cdafd71f37; no voters:
I20260927 18:41:29.916958 24545 leader_election.cc:288] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:29.916981 24548 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:29.917136 24548 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [term 1 LEADER]: Becoming Leader. State: Replica: 7e6fb6bd973f4c019e9453cdafd71f37, State: Running, Role: LEADER
I20260927 18:41:29.917260 24545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:29.917287 24548 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e6fb6bd973f4c019e9453cdafd71f37" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 39281 } }
I20260927 18:41:29.917691 24548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e6fb6bd973f4c019e9453cdafd71f37. Latest consensus state: current_term: 1 leader_uuid: "7e6fb6bd973f4c019e9453cdafd71f37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e6fb6bd973f4c019e9453cdafd71f37" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 39281 } } }
I20260927 18:41:29.917903 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 24477
I20260927 18:41:29.917915 24548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:29.918025 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:41:29.918143 24557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:29.918254 12271 external_mini_cluster.cc:1428] Setting key c59aa3b658385e5dba0fe5d2e56ab459
I20260927 18:41:29.918540 24550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e6fb6bd973f4c019e9453cdafd71f37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e6fb6bd973f4c019e9453cdafd71f37" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 39281 } } }
I20260927 18:41:29.918948 24550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:29.919180 24557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:29.919649 24545 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Sep 27 18:41:29 dist-test-slave-z47j krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:41:29.934799 24564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:29.918790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40276 (local address 127.11.251.254:39281)
0927 18:41:29.922696 (+ 3906us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:29.922727 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:29.922797 (+ 70us) server_negotiation.cc:407] Connection header received
0927 18:41:29.923104 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:29.923115 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:29.923203 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:29.923420 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:29.924304 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:29.925471 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:29.926687 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:29.927193 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:29.927327 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:29.930484 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:29.930489 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:29.930489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:29.930501 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:29.932790 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:29.933224 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:29.933227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:29.933227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:29.933330 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:29.933526 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:29.933527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:29.933527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:29.933919 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:29.934053 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:29.934316 (+ 263us) server_negotiation.cc:299] Negotiation successful
0927 18:41:29.934458 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3726,"thread_start_us":105,"threads_started":1}
I20260927 18:41:29.935839 24557 catalog_manager.cc:1347] Generated new cluster ID: 5ce52e19b8ef4b17af5ef4ae921d8d27
I20260927 18:41:29.935902 24557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:29.942914 24557 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:29.943818 24557 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:29.947571 24557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37: Generated new TSK 0
I20260927 18:41:29.947913 24557 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.251.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
I20260927 18:41:30.701351 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.251.212:47113/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:39281
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:42727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:30.818090 24590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:30.818408 24590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:30.818446 24590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:30.820192 24590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:30.820253 24590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:30.820304 24590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:41:30.822562 24590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:42727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.251.212:47113/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:39281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:30.822835 24590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:30.823173 24590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:30.829833 24598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:30.830066 24596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:30.830191 24590 server_base.cc:1034] running on GCE node
W20260927 18:41:30.830324 24595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:30.830569 24590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:30.830852 24590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:30.832026 24590 hybrid_clock.cc:648] HybridClock initialized: now 1790534490831999 us; error 44 us; skew 500 ppm
Sep 27 18:41:30 dist-test-slave-z47j krb5kdc[24033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534490, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:30.835609 24590 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:41:30.836167 24590 webserver.cc:460] Webserver started at http://127.11.251.193:36365/ using document root <none> and password file <none>
I20260927 18:41:30.836472 24590 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:30.836596 24590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:30.836766 24590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:41:30 dist-test-slave-z47j krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534490, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
I20260927 18:41:30.854368 24590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "2cf438fb48b74762a1bb3d92b1fa1c6e"
format_stamp: "Formatted at 2026-09-27 18:41:30 on dist-test-slave-z47j"
server_key: "41064d9042a5121d43dec0f686c348e3"
server_key_iv: "7c9dd6f7683385015dd2c25832bc347e"
server_key_version: "kuduclusterkey@0"
I20260927 18:41:30.854889 24590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "2cf438fb48b74762a1bb3d92b1fa1c6e"
format_stamp: "Formatted at 2026-09-27 18:41:30 on dist-test-slave-z47j"
server_key: "41064d9042a5121d43dec0f686c348e3"
server_key_iv: "7c9dd6f7683385015dd2c25832bc347e"
server_key_version: "kuduclusterkey@0"
I20260927 18:41:30.858101 24590 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260927 18:41:30.874745 24605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:30.874953 24590 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:41:30.875078 24590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "2cf438fb48b74762a1bb3d92b1fa1c6e"
format_stamp: "Formatted at 2026-09-27 18:41:30 on dist-test-slave-z47j"
server_key: "41064d9042a5121d43dec0f686c348e3"
server_key_iv: "7c9dd6f7683385015dd2c25832bc347e"
server_key_version: "kuduclusterkey@0"
I20260927 18:41:30.875205 24590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:30.899451 24590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:30.902300 24590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:30.902532 24590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:30.902872 24590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:30.903319 24590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:30.903385 24590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:30.903455 24590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:30.903509 24590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:30.911021 24590 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:39603
I20260927 18:41:30.911545 24590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:41:30.913785 24720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:39603 every 8 connection(s)
I20260927 18:41:30.918913 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 24590
I20260927 18:41:30.919044 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:41:30.919310 12271 external_mini_cluster.cc:1428] Setting key 6b2c67ba688f383769f4eadcace962c9
Sep 27 18:41:30 dist-test-slave-z47j krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534490, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.251.194@KRBTEST.COM; defaulting to no policy
I20260927 18:41:30.931052 24724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:30.914885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:38399 (local address 127.11.251.254:39281)
0927 18:41:30.920681 (+ 5796us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:30.920683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:30.920689 (+ 6us) server_negotiation.cc:407] Connection header received
0927 18:41:30.920711 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:30.920712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:30.920722 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:30.920815 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:30.921784 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:30.922591 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:30.924151 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:30.924466 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:30.924531 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:30.927427 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:30.927429 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:30.927430 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:30.927438 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:30.929307 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:30.929820 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:30.929821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:30.929822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:30.929886 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:30.930210 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:30.930211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:30.930212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:30.930367 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:30.930439 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:30.930832 (+ 393us) server_negotiation.cc:299] Negotiation successful
0927 18:41:30.930926 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5769,"thread_start_us":101,"threads_started":1}
I20260927 18:41:30.931264 24723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:30.914569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:39281 (local address 127.11.251.193:38399)
0927 18:41:30.919766 (+ 5197us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:30.919820 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:30.920455 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:30.920912 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:30.920918 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:30.921235 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:30.921646 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:30.921654 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:30.922703 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:30.922704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:30.924057 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:30.924060 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:30.924615 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:30.924616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:30.924757 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:30.925476 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:30.925487 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:30.927316 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:30.929400 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:30.929406 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:30.929409 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:30.929738 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:30.929954 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:30.929955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:30.929955 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:30.930131 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:30.930438 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:30.930445 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:30.930628 (+ 183us) client_negotiation.cc:769] Sending connection context
0927 18:41:30.930767 (+ 139us) client_negotiation.cc:240] Negotiation successful
0927 18:41:30.930992 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5141,"thread_start_us":93,"threads_started":1}
I20260927 18:41:30.932204 24721 heartbeater.cc:347] Connected to a master server at 127.11.251.254:39281
I20260927 18:41:30.932322 24721 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:30.932538 24721 heartbeater.cc:511] Master 127.11.251.254:39281 requested a full tablet report, sending...
I20260927 18:41:30.933151 24509 ts_manager.cc:194] Registered new tserver with Master: 2cf438fb48b74762a1bb3d92b1fa1c6e (127.11.251.193:39603)
I20260927 18:41:30.934091 24509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:38399
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
I20260927 18:41:30.967558 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.251.212:47113/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:39281
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:42727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:31.118984 24727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:31.119287 24727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:31.119328 24727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:31.121040 24727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:31.121097 24727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:31.121136 24727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
Sep 27 18:41:31 dist-test-slave-z47j krb5kdc[24033](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790534491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:31.123370 24727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:42727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.251.212:47113/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:39281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:31.123646 24727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:31.123974 24727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:31.130677 24737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:31.130686 24735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:31.131055 24734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:31.131666 24727 server_base.cc:1034] running on GCE node
I20260927 18:41:31.131956 24727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:31.132238 24727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:31.133404 24727 hybrid_clock.cc:648] HybridClock initialized: now 1790534491133381 us; error 36 us; skew 500 ppm
Sep 27 18:41:31 dist-test-slave-z47j krb5kdc[24033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:31.136868 24727 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:41:31.137458 24727 webserver.cc:460] Webserver started at http://127.11.251.194:34125/ using document root <none> and password file <none>
I20260927 18:41:31.137825 24727 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:31.137951 24727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:31.138144 24727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:41:31 dist-test-slave-z47j krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
I20260927 18:41:31.165936 24727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "7aed906dd8144f498fde51977e480b38"
format_stamp: "Formatted at 2026-09-27 18:41:31 on dist-test-slave-z47j"
server_key: "2247ed0bfd2cf66dd2c8eff016ab5f9a"
server_key_iv: "3a1ee1923579d6728dbb0bf93e2de044"
server_key_version: "kuduclusterkey@0"
I20260927 18:41:31.166620 24727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "7aed906dd8144f498fde51977e480b38"
format_stamp: "Formatted at 2026-09-27 18:41:31 on dist-test-slave-z47j"
server_key: "2247ed0bfd2cf66dd2c8eff016ab5f9a"
server_key_iv: "3a1ee1923579d6728dbb0bf93e2de044"
server_key_version: "kuduclusterkey@0"
I20260927 18:41:31.170007 24727 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260927 18:41:31.185730 24744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:31.186008 24727 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:41:31.186106 24727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "7aed906dd8144f498fde51977e480b38"
format_stamp: "Formatted at 2026-09-27 18:41:31 on dist-test-slave-z47j"
server_key: "2247ed0bfd2cf66dd2c8eff016ab5f9a"
server_key_iv: "3a1ee1923579d6728dbb0bf93e2de044"
server_key_version: "kuduclusterkey@0"
I20260927 18:41:31.186206 24727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:31.213106 24727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:31.215726 24727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:31.215934 24727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:31.216295 24727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:31.216722 24727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:31.216846 24727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:31.216929 24727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:31.216995 24727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:31.224200 24727 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:37093
I20260927 18:41:31.224759 24727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:41:31.226169 24858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:37093 every 8 connection(s)
I20260927 18:41:31.229075 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 24727
I20260927 18:41:31.229187 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:41:31.229400 12271 external_mini_cluster.cc:1428] Setting key 086dc721d706dc47f8e2c5da3c8175b0
Sep 27 18:41:31 dist-test-slave-z47j krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:41:31.239840 24724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.226988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:36313 (local address 127.11.251.254:39281)
0927 18:41:31.227061 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:31.227065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:31.227356 (+ 291us) server_negotiation.cc:407] Connection header received
0927 18:41:31.228122 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:31.228124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:31.228140 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:31.228172 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:31.229066 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.230683 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.232324 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.232637 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.232681 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.236363 (+ 3682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:31.236365 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:31.236365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:31.236373 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:31.238170 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:31.238693 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:31.238694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:31.238695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:31.238757 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:31.239045 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:31.239046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:31.239046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:31.239184 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:31.239227 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:31.239611 (+ 384us) server_negotiation.cc:299] Negotiation successful
0927 18:41:31.239709 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 18:41:31.240770 24861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.226986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:39281 (local address 127.11.251.194:36313)
0927 18:41:31.227208 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:31.227260 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:31.227994 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:31.228374 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:31.228378 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:31.228647 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:31.228947 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:31.228952 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.230817 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.230818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:31.232226 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:31.232229 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.232787 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.232787 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:31.232928 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.233614 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:31.233626 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:31.236258 (+ 2632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:31.238265 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:31.238272 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:31.238275 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:31.238606 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:31.238825 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:31.238825 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:31.238826 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:31.238974 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:31.239272 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:31.239279 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:31.239450 (+ 171us) client_negotiation.cc:769] Sending connection context
0927 18:41:31.240310 (+ 860us) client_negotiation.cc:240] Negotiation successful
0927 18:41:31.240501 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":108,"threads_started":1}
I20260927 18:41:31.241609 24859 heartbeater.cc:347] Connected to a master server at 127.11.251.254:39281
I20260927 18:41:31.241730 24859 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:31.241964 24859 heartbeater.cc:511] Master 127.11.251.254:39281 requested a full tablet report, sending...
I20260927 18:41:31.242494 24509 ts_manager.cc:194] Registered new tserver with Master: 7aed906dd8144f498fde51977e480b38 (127.11.251.194:37093)
I20260927 18:41:31.243029 24509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:36313
WARNING: no policy specified for kudu/127.11.251.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
I20260927 18:41:31.282629 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.251.212:47113/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:39281
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:42727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:31.413554 24864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:31.413988 24864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:31.414079 24864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:31.415967 24864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:31.416200 24864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:31.416290 24864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:41:31.418766 24864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:42727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.251.212:47113/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:39281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:31.419291 24864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:31.419711 24864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:31.426792 24870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:31.427191 24872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:31.430135 24869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:31.433192 24864 server_base.cc:1034] running on GCE node
I20260927 18:41:31.433831 24864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:31.434401 24864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:31.435617 24864 hybrid_clock.cc:648] HybridClock initialized: now 1790534491435581 us; error 73 us; skew 500 ppm
Sep 27 18:41:31 dist-test-slave-z47j krb5kdc[24033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:31.440469 24864 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:41:31.441061 24864 webserver.cc:460] Webserver started at http://127.11.251.195:45683/ using document root <none> and password file <none>
I20260927 18:41:31.441354 24864 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:31.441413 24864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:31.441551 24864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:41:31 dist-test-slave-z47j krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
I20260927 18:41:31.473426 24864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "1a2cde3b809c42a7828ad330df767eff"
format_stamp: "Formatted at 2026-09-27 18:41:31 on dist-test-slave-z47j"
server_key: "065687773c870266334a31f70c1c9838"
server_key_iv: "7654a2f52de1060d8e103d72cda99089"
server_key_version: "kuduclusterkey@0"
I20260927 18:41:31.473995 24864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "1a2cde3b809c42a7828ad330df767eff"
format_stamp: "Formatted at 2026-09-27 18:41:31 on dist-test-slave-z47j"
server_key: "065687773c870266334a31f70c1c9838"
server_key_iv: "7654a2f52de1060d8e103d72cda99089"
server_key_version: "kuduclusterkey@0"
I20260927 18:41:31.476055 24864 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:41:31.500674 24879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:31.501017 24864 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.000s sys 0.001s
I20260927 18:41:31.501161 24864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "1a2cde3b809c42a7828ad330df767eff"
format_stamp: "Formatted at 2026-09-27 18:41:31 on dist-test-slave-z47j"
server_key: "065687773c870266334a31f70c1c9838"
server_key_iv: "7654a2f52de1060d8e103d72cda99089"
server_key_version: "kuduclusterkey@0"
I20260927 18:41:31.501336 24864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:31.526033 24864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:31.528923 24864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:31.529162 24864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:31.529518 24864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:31.530050 24864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:31.530146 24864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:31.530218 24864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:31.530278 24864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:31.538033 24864 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:43679
I20260927 18:41:31.538614 24864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:41:31.544130 24992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:43679 every 8 connection(s)
I20260927 18:41:31.545648 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 24864
I20260927 18:41:31.545770 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:41:31.546013 12271 external_mini_cluster.cc:1428] Setting key 2c7cad5d16ad284c19601bdd2636b212
Sep 27 18:41:31 dist-test-slave-z47j krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:41:31.560297 24724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.545200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:46379 (local address 127.11.251.254:39281)
0927 18:41:31.545259 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:31.545262 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:31.545658 (+ 396us) server_negotiation.cc:407] Connection header received
0927 18:41:31.546506 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:31.546508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:31.546523 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:31.546558 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:31.549659 (+ 3101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.550566 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.552577 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.552894 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.552958 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.556289 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:31.556291 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:31.556291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:31.556302 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:31.558383 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:31.559064 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:31.559065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:31.559066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:31.559137 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:31.559441 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:31.559442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:31.559442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:31.559602 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:31.559657 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:31.560087 (+ 430us) server_negotiation.cc:299] Negotiation successful
0927 18:41:31.560184 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:41:31.561184 24995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.545171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:39281 (local address 127.11.251.195:46379)
0927 18:41:31.545484 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:31.545535 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:31.546369 (+ 834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:31.548870 (+ 2501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:31.548875 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:31.549185 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:31.549511 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:31.549517 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.550999 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.551000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:31.552464 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:31.552467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.553088 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.553089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:31.553247 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.554105 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:31.554117 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:31.556159 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:31.558570 (+ 2411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:31.558579 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:31.558582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:31.558972 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:31.559211 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:31.559211 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:31.559212 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:31.559363 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:31.559690 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:31.559697 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:31.559899 (+ 202us) client_negotiation.cc:769] Sending connection context
0927 18:41:31.560694 (+ 795us) client_negotiation.cc:240] Negotiation successful
0927 18:41:31.560890 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":106,"threads_started":1}
I20260927 18:41:31.562062 24993 heartbeater.cc:347] Connected to a master server at 127.11.251.254:39281
I20260927 18:41:31.562172 24993 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:31.562389 24993 heartbeater.cc:511] Master 127.11.251.254:39281 requested a full tablet report, sending...
I20260927 18:41:31.562870 24509 ts_manager.cc:194] Registered new tserver with Master: 1a2cde3b809c42a7828ad330df767eff (127.11.251.195:43679)
I20260927 18:41:31.563416 24509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:46379
I20260927 18:41:31.570302 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:41:31.571146 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 24590
I20260927 18:41:31.577885 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 24727
I20260927 18:41:31.584329 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 24864
I20260927 18:41:31.596855 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 24477
I20260927 18:41:31.604592 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:39281
--webserver_interface=127.11.251.254
--webserver_port=40215
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:42727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:39281
--ranger_config_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.251.212:47113/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.251.212:47113/kms with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:31.698256 24996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:31.698525 24996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:31.698562 24996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:31.699751 24996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:31.699823 24996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:31.699851 24996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:31.699883 24996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:31.699906 24996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:31.701695 24996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:42727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.251.212:47113/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:39281
--ranger_config_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:39281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=40215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:31.701985 24996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:31.702229 24996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:31.707630 25001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:31.707775 24996 server_base.cc:1034] running on GCE node
W20260927 18:41:31.707633 25002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:31.707667 25004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:31.708259 24996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:31.708613 24996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:31.709992 24996 hybrid_clock.cc:648] HybridClock initialized: now 1790534491709963 us; error 46 us; skew 500 ppm
Sep 27 18:41:31 dist-test-slave-z47j krb5kdc[24033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:31.712747 24996 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:41:31.713299 24996 webserver.cc:460] Webserver started at http://127.11.251.254:40215/ using document root <none> and password file <none>
I20260927 18:41:31.713543 24996 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:31.713642 24996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 18:41:31 dist-test-slave-z47j krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
I20260927 18:41:31.777031 24996 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 18:41:31.777944 25012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:31.778112 24996 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:41:31.778236 24996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "7e6fb6bd973f4c019e9453cdafd71f37"
format_stamp: "Formatted at 2026-09-27 18:41:29 on dist-test-slave-z47j"
server_key: "efb0899c721274779025cff8cf409e73"
server_key_iv: "71de4fab8f2382dd55f48a6c2844545e"
server_key_version: "kuduclusterkey@0"
I20260927 18:41:31.778687 24996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:31.793653 24996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:31.796013 24996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:31.796188 24996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:31.801909 24996 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:39281
I20260927 18:41:31.801991 25064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:39281 every 8 connection(s)
I20260927 18:41:31.802513 24996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:31.803999 25065 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:41:31.804778 25065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37: Bootstrap starting.
I20260927 18:41:31.812767 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 24996
I20260927 18:41:31.813287 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.251.212:47113/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:39603
--local_ip_for_outbound_sockets=127.11.251.193
--tserver_master_addrs=127.11.251.254:39281
--webserver_port=36365
--webserver_interface=127.11.251.193
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:42727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.251.212:47113/kms with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:41:31.816169 25065 log.cc:824] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:31.818519 25065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:41:31.818854 25065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37: Bootstrap complete.
I20260927 18:41:31.821625 25065 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e6fb6bd973f4c019e9453cdafd71f37" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 39281 } }
I20260927 18:41:31.821957 25065 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e6fb6bd973f4c019e9453cdafd71f37, State: Initialized, Role: FOLLOWER
I20260927 18:41:31.822207 25065 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e6fb6bd973f4c019e9453cdafd71f37" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 39281 } }
I20260927 18:41:31.822304 25065 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:31.822379 25065 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:31.822458 25065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [term 1 FOLLOWER]: Advancing to term 2
I20260927 18:41:31.823637 25065 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e6fb6bd973f4c019e9453cdafd71f37" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 39281 } }
I20260927 18:41:31.823773 25065 leader_election.cc:302] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e6fb6bd973f4c019e9453cdafd71f37; no voters:
I20260927 18:41:31.823992 25065 leader_election.cc:288] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 18:41:31.824031 25069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 18:41:31.824270 25069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [term 2 LEADER]: Becoming Leader. State: Replica: 7e6fb6bd973f4c019e9453cdafd71f37, State: Running, Role: LEADER
I20260927 18:41:31.824402 25065 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:31.824455 25069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e6fb6bd973f4c019e9453cdafd71f37" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 39281 } }
I20260927 18:41:31.824793 25069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7e6fb6bd973f4c019e9453cdafd71f37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e6fb6bd973f4c019e9453cdafd71f37" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 39281 } } }
I20260927 18:41:31.824945 25069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:31.825150 25069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e6fb6bd973f4c019e9453cdafd71f37. Latest consensus state: current_term: 2 leader_uuid: "7e6fb6bd973f4c019e9453cdafd71f37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e6fb6bd973f4c019e9453cdafd71f37" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 39281 } } }
I20260927 18:41:31.825263 25069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:31.825430 25077 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:31.826138 25077 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:31.826556 25065 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260927 18:41:31.826556 25077 catalog_manager.cc:1259] Loaded cluster ID: 5ce52e19b8ef4b17af5ef4ae921d8d27
I20260927 18:41:31.827224 25077 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:31.828497 25077 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:31.828776 25077 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37: Loaded TSK: 0
I20260927 18:41:31.829063 25077 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 18:41:31.940544 25067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:31.940960 25067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:31.941061 25067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:31.942919 25067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:31.943132 25067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:31.943217 25067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:41:31.945288 25067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:42727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.251.212:47113/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:39603
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=36365
--tserver_master_addrs=127.11.251.254:39281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:31.945883 25067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:31.946197 25067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:31.953354 25103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:31.953603 25105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:31.953704 25102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:31.954731 25067 server_base.cc:1034] running on GCE node
I20260927 18:41:31.955071 25067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:31.955364 25067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:31.956543 25067 hybrid_clock.cc:648] HybridClock initialized: now 1790534491956508 us; error 48 us; skew 500 ppm
Sep 27 18:41:31 dist-test-slave-z47j krb5kdc[24033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:31.960142 25067 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:41:31.960716 25067 webserver.cc:460] Webserver started at http://127.11.251.193:36365/ using document root <none> and password file <none>
I20260927 18:41:31.960999 25067 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:31.961068 25067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 18:41:31 dist-test-slave-z47j krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
I20260927 18:41:31.983800 25067 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260927 18:41:31.986485 25113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:31.986737 25067 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:41:31.986835 25067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "2cf438fb48b74762a1bb3d92b1fa1c6e"
format_stamp: "Formatted at 2026-09-27 18:41:30 on dist-test-slave-z47j"
server_key: "41064d9042a5121d43dec0f686c348e3"
server_key_iv: "7c9dd6f7683385015dd2c25832bc347e"
server_key_version: "kuduclusterkey@0"
I20260927 18:41:31.987399 25067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:32.016578 25067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:32.019361 25067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:32.019613 25067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:32.019936 25067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:32.020427 25067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:32.020537 25067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:32.020656 25067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:32.020736 25067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:32.028532 25067 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:39603
I20260927 18:41:32.029130 25067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:41:32.032884 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 25067
I20260927 18:41:32.033226 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.251.212:47113/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:37093
--local_ip_for_outbound_sockets=127.11.251.194
--tserver_master_addrs=127.11.251.254:39281
--webserver_port=34125
--webserver_interface=127.11.251.194
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:42727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.251.212:47113/kms with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:41:32.040315 25226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:39603 every 8 connection(s)
Sep 27 18:41:32 dist-test-slave-z47j krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:41:32.066116 25231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.041563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:33339 (local address 127.11.251.254:39281)
0927 18:41:32.047154 (+ 5591us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:32.047166 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:32.047226 (+ 60us) server_negotiation.cc:407] Connection header received
0927 18:41:32.048159 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:32.048166 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:32.048234 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:32.048400 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:32.049398 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.050903 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.052491 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.053107 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.053216 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.058118 (+ 4902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:32.058123 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:32.058124 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:32.058137 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:32.063085 (+ 4948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:32.064352 (+ 1267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:32.064356 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:32.064356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:32.064499 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:32.064831 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:32.064832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:32.064833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:32.065175 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:32.065291 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:32.065721 (+ 430us) server_negotiation.cc:299] Negotiation successful
0927 18:41:32.065871 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5506,"thread_start_us":88,"threads_started":1}
I20260927 18:41:32.066668 25230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.041165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:39281 (local address 127.11.251.193:33339)
0927 18:41:32.046312 (+ 5147us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:32.046379 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:32.047799 (+ 1420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:32.048595 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:32.048600 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:32.048894 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:32.049219 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.049225 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.051032 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.051033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.052387 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.052391 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.053347 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.053348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.053494 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.055881 (+ 2387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:32.055893 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:32.057882 (+ 1989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:32.063845 (+ 5963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:32.063868 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:32.063871 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:32.064229 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:32.064589 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:32.064590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:32.064590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:32.064733 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:32.065325 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:32.065333 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:32.065500 (+ 167us) client_negotiation.cc:769] Sending connection context
0927 18:41:32.066258 (+ 758us) client_negotiation.cc:240] Negotiation successful
0927 18:41:32.066455 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5078,"thread_start_us":130,"threads_started":1}
I20260927 18:41:32.067625 25227 heartbeater.cc:347] Connected to a master server at 127.11.251.254:39281
I20260927 18:41:32.067724 25227 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:32.067924 25227 heartbeater.cc:511] Master 127.11.251.254:39281 requested a full tablet report, sending...
W20260927 18:41:32.068943 25227 heartbeater.cc:644] Failed to heartbeat to 127.11.251.254:39281 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 18:41:32.069010 25227 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:32.069065 25227 heartbeater.cc:511] Master 127.11.251.254:39281 requested a full tablet report, sending...
I20260927 18:41:32.069545 25227 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:32.069656 25227 heartbeater.cc:511] Master 127.11.251.254:39281 requested a full tablet report, sending...
W20260927 18:41:32.070003 25227 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 18:41:32.153972 25229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:32.154363 25229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:32.154470 25229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:32.156144 25229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:32.156263 25229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:32.156348 25229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:41:32.158593 25229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:42727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.251.212:47113/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:37093
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=34125
--tserver_master_addrs=127.11.251.254:39281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:32.159089 25229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:32.159412 25229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:32.165930 25239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:32.166077 25236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:32.166178 25237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:32.167371 25229 server_base.cc:1034] running on GCE node
I20260927 18:41:32.167606 25229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:32.167872 25229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:32.169031 25229 hybrid_clock.cc:648] HybridClock initialized: now 1790534492169008 us; error 42 us; skew 500 ppm
Sep 27 18:41:32 dist-test-slave-z47j krb5kdc[24033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534492, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:32.171996 25229 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:41:32.172550 25229 webserver.cc:460] Webserver started at http://127.11.251.194:34125/ using document root <none> and password file <none>
I20260927 18:41:32.172894 25229 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:32.173025 25229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 18:41:32 dist-test-slave-z47j krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534492, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
I20260927 18:41:32.197956 25229 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:41:32.199988 25247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:32.200230 25229 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:41:32.200367 25229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "7aed906dd8144f498fde51977e480b38"
format_stamp: "Formatted at 2026-09-27 18:41:31 on dist-test-slave-z47j"
server_key: "2247ed0bfd2cf66dd2c8eff016ab5f9a"
server_key_iv: "3a1ee1923579d6728dbb0bf93e2de044"
server_key_version: "kuduclusterkey@0"
I20260927 18:41:32.201721 25229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:32.229888 25229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:32.232465 25229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:32.232653 25229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:32.232976 25229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:32.233408 25229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:32.233465 25229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:32.233511 25229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:32.233538 25229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:32.241925 25229 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:37093
I20260927 18:41:32.242513 25229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:41:32.243197 25360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:37093 every 8 connection(s)
Sep 27 18:41:32 dist-test-slave-z47j krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534492, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:41:32.252696 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 25229
I20260927 18:41:32.253049 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.251.212:47113/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:43679
--local_ip_for_outbound_sockets=127.11.251.195
--tserver_master_addrs=127.11.251.254:39281
--webserver_port=45683
--webserver_interface=127.11.251.195
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:42727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.251.212:47113/kms with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:41:32.254041 25231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.243469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:60879 (local address 127.11.251.254:39281)
0927 18:41:32.243522 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:32.243525 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:32.243824 (+ 299us) server_negotiation.cc:407] Connection header received
0927 18:41:32.244459 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:32.244461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:32.244480 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:32.244532 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:32.245169 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.245943 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.247014 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.247253 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.247411 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.250113 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:32.250116 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:32.250116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:32.250133 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:32.252061 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:32.252628 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:32.252629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:32.252630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:32.252705 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:32.253036 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:32.253037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:32.253037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:32.253217 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:32.253342 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:32.253801 (+ 459us) server_negotiation.cc:299] Negotiation successful
0927 18:41:32.253919 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:41:32.255115 25363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.243505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:39281 (local address 127.11.251.194:60879)
0927 18:41:32.243677 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:32.243717 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:32.244311 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:32.244622 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:32.244627 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:32.244829 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:32.245051 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.245056 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.246049 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.246050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.246910 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.246913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.247468 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.247469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.247604 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.248291 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:32.248309 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:32.249979 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:32.252174 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:32.252182 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:32.252184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:32.252514 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:32.252794 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:32.252795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:32.252795 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:32.252942 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:32.253399 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:32.253408 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:32.253614 (+ 206us) client_negotiation.cc:769] Sending connection context
0927 18:41:32.254573 (+ 959us) client_negotiation.cc:240] Negotiation successful
0927 18:41:32.254807 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
I20260927 18:41:32.255942 25361 heartbeater.cc:347] Connected to a master server at 127.11.251.254:39281
I20260927 18:41:32.256088 25361 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:32.256349 25361 heartbeater.cc:511] Master 127.11.251.254:39281 requested a full tablet report, sending...
W20260927 18:41:32.257120 25361 heartbeater.cc:644] Failed to heartbeat to 127.11.251.254:39281 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 18:41:32.257225 25361 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:32.257324 25361 heartbeater.cc:511] Master 127.11.251.254:39281 requested a full tablet report, sending...
I20260927 18:41:32.257720 25361 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:32.257826 25361 heartbeater.cc:511] Master 127.11.251.254:39281 requested a full tablet report, sending...
W20260927 18:41:32.258225 25361 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 18:41:32.379405 25364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:32.379734 25364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:32.379781 25364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:32.381647 25364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:32.381816 25364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:32.381920 25364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:41:32.384438 25364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:42727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.251.212:47113/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:43679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=45683
--tserver_master_addrs=127.11.251.254:39281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:32.384788 25364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:32.385198 25364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:32.392660 25372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:32.393474 25364 server_base.cc:1034] running on GCE node
W20260927 18:41:32.393712 25369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:32.393971 25370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:32.394230 25364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:32.394568 25364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:32.395763 25364 hybrid_clock.cc:648] HybridClock initialized: now 1790534492395731 us; error 47 us; skew 500 ppm
Sep 27 18:41:32 dist-test-slave-z47j krb5kdc[24033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534492, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:32.399567 25364 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:41:32.400252 25364 webserver.cc:460] Webserver started at http://127.11.251.195:45683/ using document root <none> and password file <none>
I20260927 18:41:32.400635 25364 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:32.400784 25364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 18:41:32 dist-test-slave-z47j krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534492, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
I20260927 18:41:32.423600 25364 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:41:32.425684 25381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:32.425904 25364 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:41:32.426012 25364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "1a2cde3b809c42a7828ad330df767eff"
format_stamp: "Formatted at 2026-09-27 18:41:31 on dist-test-slave-z47j"
server_key: "065687773c870266334a31f70c1c9838"
server_key_iv: "7654a2f52de1060d8e103d72cda99089"
server_key_version: "kuduclusterkey@0"
I20260927 18:41:32.426611 25364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:32.441799 25364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:32.444145 25364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:32.444332 25364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:32.444628 25364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:32.445125 25364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:32.445185 25364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:32.445256 25364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:32.445286 25364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:32.453649 25364 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:43679
I20260927 18:41:32.453737 25494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:43679 every 8 connection(s)
I20260927 18:41:32.454180 25364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:41:32.461046 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 25364
Sep 27 18:41:32 dist-test-slave-z47j krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534492, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:41:32.465297 25231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.455087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:36681 (local address 127.11.251.254:39281)
0927 18:41:32.455158 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:32.455160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:32.455418 (+ 258us) server_negotiation.cc:407] Connection header received
0927 18:41:32.456152 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:32.456154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:32.456167 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:32.456227 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:32.456991 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.457623 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.458779 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.459006 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.459095 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.461791 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:32.461793 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:32.461793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:32.461802 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:32.463553 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:32.464068 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:32.464069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:32.464070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:32.464142 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:32.464469 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:32.464470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:32.464470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:32.464610 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:32.464682 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:32.465090 (+ 408us) server_negotiation.cc:299] Negotiation successful
0927 18:41:32.465185 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 18:41:32.465952 25497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.455054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:39281 (local address 127.11.251.195:36681)
0927 18:41:32.455254 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:32.455304 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:32.456030 (+ 726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:32.456306 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:32.456311 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:32.456562 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:32.456860 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.456877 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.457709 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.457710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.458674 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.458677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.459130 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.459131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.459259 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.459869 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:32.459878 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:32.461648 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:32.463634 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:32.463641 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:32.463644 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:32.463987 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:32.464221 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:32.464222 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:32.464223 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:32.464387 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:32.464693 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:32.464699 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:32.464901 (+ 202us) client_negotiation.cc:769] Sending connection context
0927 18:41:32.465452 (+ 551us) client_negotiation.cc:240] Negotiation successful
0927 18:41:32.465667 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":95,"threads_started":1}
I20260927 18:41:32.466543 25495 heartbeater.cc:347] Connected to a master server at 127.11.251.254:39281
I20260927 18:41:32.466660 25495 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:32.466876 25495 heartbeater.cc:511] Master 127.11.251.254:39281 requested a full tablet report, sending...
W20260927 18:41:32.467840 25495 heartbeater.cc:644] Failed to heartbeat to 127.11.251.254:39281 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 18:41:32.467931 25495 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:32.468011 25495 heartbeater.cc:511] Master 127.11.251.254:39281 requested a full tablet report, sending...
I20260927 18:41:32.468408 25495 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:32.468814 25495 heartbeater.cc:511] Master 127.11.251.254:39281 requested a full tablet report, sending...
W20260927 18:41:32.469251 25495 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 18:41:32.470814 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.461650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40280 (local address 127.11.251.254:39281)
0927 18:41:32.461933 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:32.461935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:32.461943 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:41:32.461962 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:32.461962 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:32.461977 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:32.462053 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:32.463151 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.463897 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.464929 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.465154 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.465238 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.467149 (+ 1911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:32.467150 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:32.467151 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:32.467159 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:32.469449 (+ 2290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:32.469861 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:32.469862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:32.469863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:32.469927 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:32.470143 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:32.470143 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:32.470144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:32.470266 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:32.470476 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:32.470526 (+ 50us) server_negotiation.cc:299] Negotiation successful
0927 18:41:32.470616 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":70,"threads_started":1}
Sep 27 18:41:33 dist-test-slave-z47j krb5kdc[24033](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790534493, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:33.070212 25227 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:33.070427 25227 heartbeater.cc:511] Master 127.11.251.254:39281 requested a full tablet report, sending...
I20260927 18:41:33.070951 25029 ts_manager.cc:194] Registered new tserver with Master: 2cf438fb48b74762a1bb3d92b1fa1c6e (127.11.251.193:39603)
I20260927 18:41:33.072114 25029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:33339
I20260927 18:41:33.258486 25361 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:33.258713 25361 heartbeater.cc:511] Master 127.11.251.254:39281 requested a full tablet report, sending...
I20260927 18:41:33.259311 25029 ts_manager.cc:194] Registered new tserver with Master: 7aed906dd8144f498fde51977e480b38 (127.11.251.194:37093)
I20260927 18:41:33.259964 25029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:60879
Sep 27 18:41:33 dist-test-slave-z47j krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534493, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
I20260927 18:41:33.469440 25495 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:33.469691 25495 heartbeater.cc:511] Master 127.11.251.254:39281 requested a full tablet report, sending...
I20260927 18:41:33.470166 25029 ts_manager.cc:194] Registered new tserver with Master: 1a2cde3b809c42a7828ad330df767eff (127.11.251.195:43679)
I20260927 18:41:33.470755 25029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:36681
I20260927 18:41:33.472906 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:41:33.486344 25516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:33.474058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40296 (local address 127.11.251.254:39281)
0927 18:41:33.477704 (+ 3646us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:33.477707 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:33.477848 (+ 141us) server_negotiation.cc:407] Connection header received
0927 18:41:33.477960 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:33.477961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:33.477975 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:33.478038 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:33.478875 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.479713 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.480791 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.481118 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.481172 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:33.483091 (+ 1919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:33.483093 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:33.483093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:33.483101 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:33.484961 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:33.485393 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:33.485394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:33.485394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:33.485456 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:33.485705 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:33.485706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:33.485707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:33.485848 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:33.485926 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:33.486146 (+ 220us) server_negotiation.cc:299] Negotiation successful
0927 18:41:33.486236 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3615,"thread_start_us":86,"threads_started":1}
I20260927 18:41:33.489559 25029 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40296:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:41:33.492658 25029 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:41:33.508986 25527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:33.500712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:39603 (local address 127.0.0.1:48736)
0927 18:41:33.502017 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:33.502025 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:33.502067 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:33.502374 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:33.502378 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:33.502381 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:33.502462 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:33.502465 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.504198 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.504199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:33.507212 (+ 3013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:33.507215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.508636 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.508636 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:33.508734 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:33.508749 (+ 15us) client_negotiation.cc:769] Sending connection context
0927 18:41:33.508818 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 18:41:33.508883 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1291,"thread_start_us":75,"threads_started":1}
I20260927 18:41:33.509606 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:33.500845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48736 (local address 127.11.251.193:39603)
0927 18:41:33.501684 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:33.501690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:33.502120 (+ 430us) server_negotiation.cc:407] Connection header received
0927 18:41:33.502168 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:33.502171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:33.502189 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:33.502260 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:33.502541 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.504096 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.507309 (+ 3213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.508177 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.508526 (+ 349us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:33.508602 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:33.509445 (+ 843us) server_negotiation.cc:299] Negotiation successful
0927 18:41:33.509499 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":791,"thread_start_us":69,"threads_started":1}
I20260927 18:41:33.513052 25161 tablet_service.cc:1467] Processing CreateTablet for tablet f038b840f67c4103a8cd36bbde70ed00 (DEFAULT_TABLE table=test-table [id=293754cd037f4d4dae4a27b700e1c3ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:33.513526 25161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f038b840f67c4103a8cd36bbde70ed00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:33.515349 25525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:33.500925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:37093 (local address 127.0.0.1:36406)
0927 18:41:33.505768 (+ 4843us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:33.505778 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:33.505828 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:33.506252 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:33.506253 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:33.506255 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:33.506298 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:33.506300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.510541 (+ 4241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.510542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:33.512145 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:33.512148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.515105 (+ 2957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.515107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:33.515165 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:33.515171 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:41:33.515225 (+ 54us) client_negotiation.cc:240] Negotiation successful
0927 18:41:33.515269 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4822,"thread_start_us":77,"threads_started":1}
I20260927 18:41:33.515654 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:33.501174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36406 (local address 127.11.251.194:37093)
0927 18:41:33.506005 (+ 4831us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:33.506010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:33.506020 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:41:33.506059 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:33.506061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:33.506079 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:33.506152 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:33.506409 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.510442 (+ 4033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.512998 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.514929 (+ 1931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.515010 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:33.515068 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:33.515524 (+ 456us) server_negotiation.cc:299] Negotiation successful
0927 18:41:33.515572 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4790,"thread_start_us":57,"threads_started":1}
I20260927 18:41:33.516564 25295 tablet_service.cc:1467] Processing CreateTablet for tablet f038b840f67c4103a8cd36bbde70ed00 (DEFAULT_TABLE table=test-table [id=293754cd037f4d4dae4a27b700e1c3ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:33.517015 25295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f038b840f67c4103a8cd36bbde70ed00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:33.518781 25523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:33.500451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:43679 (local address 127.0.0.1:56320)
0927 18:41:33.501785 (+ 1334us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:33.501829 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:33.501913 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:33.512874 (+ 10961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:33.512875 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:33.512877 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:33.512919 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:33.512921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.516203 (+ 3282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.516204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:33.517263 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:33.517266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.518542 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.518543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:33.518623 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:33.518628 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:41:33.518663 (+ 35us) client_negotiation.cc:240] Negotiation successful
0927 18:41:33.518703 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1302,"thread_start_us":82,"threads_started":1}
I20260927 18:41:33.519493 25524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:33.500592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56320 (local address 127.11.251.195:43679)
0927 18:41:33.512600 (+ 12008us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:33.512605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:33.512617 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:41:33.512654 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:33.512657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:33.512675 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:33.512756 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:33.513659 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.514638 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.517375 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.518293 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.519234 (+ 941us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:33.519309 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:33.519351 (+ 42us) server_negotiation.cc:299] Negotiation successful
0927 18:41:33.519402 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11958,"thread_start_us":83,"threads_started":1}
I20260927 18:41:33.520109 25529 tablet_bootstrap.cc:492] T f038b840f67c4103a8cd36bbde70ed00 P 2cf438fb48b74762a1bb3d92b1fa1c6e: Bootstrap starting.
I20260927 18:41:33.520700 25429 tablet_service.cc:1467] Processing CreateTablet for tablet f038b840f67c4103a8cd36bbde70ed00 (DEFAULT_TABLE table=test-table [id=293754cd037f4d4dae4a27b700e1c3ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:33.521116 25429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f038b840f67c4103a8cd36bbde70ed00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:33.521409 25529 tablet_bootstrap.cc:654] T f038b840f67c4103a8cd36bbde70ed00 P 2cf438fb48b74762a1bb3d92b1fa1c6e: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:33.521785 25529 log.cc:824] T f038b840f67c4103a8cd36bbde70ed00 P 2cf438fb48b74762a1bb3d92b1fa1c6e: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:33.522655 25529 tablet_bootstrap.cc:492] T f038b840f67c4103a8cd36bbde70ed00 P 2cf438fb48b74762a1bb3d92b1fa1c6e: No bootstrap required, opened a new log
I20260927 18:41:33.522722 25529 ts_tablet_manager.cc:1397] T f038b840f67c4103a8cd36bbde70ed00 P 2cf438fb48b74762a1bb3d92b1fa1c6e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:41:33.525408 25529 raft_consensus.cc:348] T f038b840f67c4103a8cd36bbde70ed00 P 2cf438fb48b74762a1bb3d92b1fa1c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2cde3b809c42a7828ad330df767eff" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 43679 } } peers { permanent_uuid: "7aed906dd8144f498fde51977e480b38" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37093 } } peers { permanent_uuid: "2cf438fb48b74762a1bb3d92b1fa1c6e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39603 } }
I20260927 18:41:33.525784 25529 raft_consensus.cc:374] T f038b840f67c4103a8cd36bbde70ed00 P 2cf438fb48b74762a1bb3d92b1fa1c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:33.525872 25529 raft_consensus.cc:729] T f038b840f67c4103a8cd36bbde70ed00 P 2cf438fb48b74762a1bb3d92b1fa1c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cf438fb48b74762a1bb3d92b1fa1c6e, State: Initialized, Role: FOLLOWER
I20260927 18:41:33.526089 25529 consensus_queue.cc:260] T f038b840f67c4103a8cd36bbde70ed00 P 2cf438fb48b74762a1bb3d92b1fa1c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2cde3b809c42a7828ad330df767eff" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 43679 } } peers { permanent_uuid: "7aed906dd8144f498fde51977e480b38" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37093 } } peers { permanent_uuid: "2cf438fb48b74762a1bb3d92b1fa1c6e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39603 } }
I20260927 18:41:33.526485 25529 ts_tablet_manager.cc:1428] T f038b840f67c4103a8cd36bbde70ed00 P 2cf438fb48b74762a1bb3d92b1fa1c6e: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260927 18:41:33.526659 25227 heartbeater.cc:503] Master 127.11.251.254:39281 was elected leader, sending a full tablet report...
I20260927 18:41:33.528472 25532 tablet_bootstrap.cc:492] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff: Bootstrap starting.
I20260927 18:41:33.529263 25532 tablet_bootstrap.cc:654] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:33.529711 25532 log.cc:824] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:33.530546 25532 tablet_bootstrap.cc:492] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff: No bootstrap required, opened a new log
I20260927 18:41:33.530612 25532 ts_tablet_manager.cc:1397] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:41:33.531075 25530 tablet_bootstrap.cc:492] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38: Bootstrap starting.
I20260927 18:41:33.531877 25530 tablet_bootstrap.cc:654] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:33.532186 25530 log.cc:824] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:33.533275 25532 raft_consensus.cc:348] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2cde3b809c42a7828ad330df767eff" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 43679 } } peers { permanent_uuid: "7aed906dd8144f498fde51977e480b38" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37093 } } peers { permanent_uuid: "2cf438fb48b74762a1bb3d92b1fa1c6e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39603 } }
I20260927 18:41:33.533455 25532 raft_consensus.cc:374] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:33.533486 25532 raft_consensus.cc:729] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a2cde3b809c42a7828ad330df767eff, State: Initialized, Role: FOLLOWER
I20260927 18:41:33.533761 25532 consensus_queue.cc:260] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2cde3b809c42a7828ad330df767eff" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 43679 } } peers { permanent_uuid: "7aed906dd8144f498fde51977e480b38" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37093 } } peers { permanent_uuid: "2cf438fb48b74762a1bb3d92b1fa1c6e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39603 } }
I20260927 18:41:33.534127 25532 ts_tablet_manager.cc:1428] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:41:33.534283 25495 heartbeater.cc:503] Master 127.11.251.254:39281 was elected leader, sending a full tablet report...
I20260927 18:41:33.535382 25530 tablet_bootstrap.cc:492] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38: No bootstrap required, opened a new log
I20260927 18:41:33.535529 25530 ts_tablet_manager.cc:1397] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260927 18:41:33.537982 25530 raft_consensus.cc:348] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2cde3b809c42a7828ad330df767eff" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 43679 } } peers { permanent_uuid: "7aed906dd8144f498fde51977e480b38" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37093 } } peers { permanent_uuid: "2cf438fb48b74762a1bb3d92b1fa1c6e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39603 } }
I20260927 18:41:33.538197 25530 raft_consensus.cc:374] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:33.538236 25530 raft_consensus.cc:729] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aed906dd8144f498fde51977e480b38, State: Initialized, Role: FOLLOWER
I20260927 18:41:33.538439 25530 consensus_queue.cc:260] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2cde3b809c42a7828ad330df767eff" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 43679 } } peers { permanent_uuid: "7aed906dd8144f498fde51977e480b38" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37093 } } peers { permanent_uuid: "2cf438fb48b74762a1bb3d92b1fa1c6e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39603 } }
I20260927 18:41:33.538763 25530 ts_tablet_manager.cc:1428] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:41:33.538827 25361 heartbeater.cc:503] Master 127.11.251.254:39281 was elected leader, sending a full tablet report...
W20260927 18:41:33.543044 25228 tablet.cc:2369] T f038b840f67c4103a8cd36bbde70ed00 P 2cf438fb48b74762a1bb3d92b1fa1c6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 27 18:41:33 dist-test-slave-z47j krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534493, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for HTTP/127.11.251.212@KRBTEST.COM
W20260927 18:41:33.705844 25496 tablet.cc:2369] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:41:33.744243 25362 tablet.cc:2369] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:41:33.759370 25537 raft_consensus.cc:482] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:41:33.759620 25537 raft_consensus.cc:504] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2cde3b809c42a7828ad330df767eff" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 43679 } } peers { permanent_uuid: "7aed906dd8144f498fde51977e480b38" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37093 } } peers { permanent_uuid: "2cf438fb48b74762a1bb3d92b1fa1c6e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39603 } }
I20260927 18:41:33.760129 25537 leader_election.cc:288] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a2cde3b809c42a7828ad330df767eff (127.11.251.195:43679), 2cf438fb48b74762a1bb3d92b1fa1c6e (127.11.251.193:39603)
I20260927 18:41:33.768235 25524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:33.761284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:51867 (local address 127.11.251.195:43679)
0927 18:41:33.761333 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:33.761335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:33.764432 (+ 3097us) server_negotiation.cc:407] Connection header received
0927 18:41:33.764541 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:33.764542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:33.764554 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:33.764583 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:33.764824 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.765606 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.767172 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.767795 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.767851 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:33.767864 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:33.768097 (+ 233us) server_negotiation.cc:299] Negotiation successful
0927 18:41:33.768146 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 18:41:33.768790 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:33.761064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:43679 (local address 127.11.251.194:51867)
0927 18:41:33.764351 (+ 3287us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:33.764364 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:33.764463 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:33.764654 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:33.764655 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:33.764658 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:33.764748 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:33.764749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.765713 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.765713 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:33.767068 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:33.767071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.767949 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.767950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:33.768019 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:33.768023 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:41:33.768629 (+ 606us) client_negotiation.cc:240] Negotiation successful
0927 18:41:33.768681 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3264,"thread_start_us":97,"threads_started":1}
I20260927 18:41:33.769325 25445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f038b840f67c4103a8cd36bbde70ed00" candidate_uuid: "7aed906dd8144f498fde51977e480b38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a2cde3b809c42a7828ad330df767eff" is_pre_election: true
I20260927 18:41:33.769524 25445 raft_consensus.cc:2454] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aed906dd8144f498fde51977e480b38 in term 0.
I20260927 18:41:33.769899 25251 leader_election.cc:302] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a2cde3b809c42a7828ad330df767eff, 7aed906dd8144f498fde51977e480b38; no voters:
I20260927 18:41:33.770041 25537 raft_consensus.cc:2781] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:41:33.770081 25537 raft_consensus.cc:482] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:41:33.770103 25537 raft_consensus.cc:3037] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:33.771158 25537 raft_consensus.cc:504] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2cde3b809c42a7828ad330df767eff" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 43679 } } peers { permanent_uuid: "7aed906dd8144f498fde51977e480b38" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37093 } } peers { permanent_uuid: "2cf438fb48b74762a1bb3d92b1fa1c6e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39603 } }
I20260927 18:41:33.771318 25537 leader_election.cc:288] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 [CANDIDATE]: Term 1 election: Requested vote from peers 1a2cde3b809c42a7828ad330df767eff (127.11.251.195:43679), 2cf438fb48b74762a1bb3d92b1fa1c6e (127.11.251.193:39603)
I20260927 18:41:33.771690 25445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f038b840f67c4103a8cd36bbde70ed00" candidate_uuid: "7aed906dd8144f498fde51977e480b38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a2cde3b809c42a7828ad330df767eff"
I20260927 18:41:33.771771 25445 raft_consensus.cc:3037] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:33.772774 25445 raft_consensus.cc:2454] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aed906dd8144f498fde51977e480b38 in term 1.
I20260927 18:41:33.773044 25251 leader_election.cc:302] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a2cde3b809c42a7828ad330df767eff, 7aed906dd8144f498fde51977e480b38; no voters:
I20260927 18:41:33.773219 25537 raft_consensus.cc:2781] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:33.773387 25537 raft_consensus.cc:686] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 [term 1 LEADER]: Becoming Leader. State: Replica: 7aed906dd8144f498fde51977e480b38, State: Running, Role: LEADER
I20260927 18:41:33.773485 25537 consensus_queue.cc:237] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2cde3b809c42a7828ad330df767eff" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 43679 } } peers { permanent_uuid: "7aed906dd8144f498fde51977e480b38" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37093 } } peers { permanent_uuid: "2cf438fb48b74762a1bb3d92b1fa1c6e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39603 } }
I20260927 18:41:33.774497 25029 catalog_manager.cc:5495] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aed906dd8144f498fde51977e480b38 (127.11.251.194). New cstate: current_term: 1 leader_uuid: "7aed906dd8144f498fde51977e480b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2cde3b809c42a7828ad330df767eff" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 43679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aed906dd8144f498fde51977e480b38" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2cf438fb48b74762a1bb3d92b1fa1c6e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39603 } health_report { overall_health: UNKNOWN } } }
I20260927 18:41:33.778853 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:33.760667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:60895 (local address 127.11.251.193:39603)
0927 18:41:33.760737 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:33.760740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:33.760774 (+ 34us) server_negotiation.cc:407] Connection header received
0927 18:41:33.760816 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:33.760817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:33.760831 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:33.760902 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:33.761120 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.761938 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.777737 (+ 15799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.778393 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.778453 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:33.778467 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:33.778708 (+ 241us) server_negotiation.cc:299] Negotiation successful
0927 18:41:33.778770 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 18:41:33.779359 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:33.760356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:39603 (local address 127.11.251.194:60895)
0927 18:41:33.760636 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:33.760652 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:33.760714 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:33.760916 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:33.760917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:33.760919 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:33.761020 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:33.761023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.762048 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.762049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:33.775712 (+ 13663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:33.775715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.778534 (+ 2819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.778535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:33.778613 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:33.778618 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:41:33.779204 (+ 586us) client_negotiation.cc:240] Negotiation successful
0927 18:41:33.779261 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":188,"threads_started":1}
I20260927 18:41:33.779903 25181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f038b840f67c4103a8cd36bbde70ed00" candidate_uuid: "7aed906dd8144f498fde51977e480b38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cf438fb48b74762a1bb3d92b1fa1c6e" is_pre_election: true
I20260927 18:41:33.780093 25181 raft_consensus.cc:2454] T f038b840f67c4103a8cd36bbde70ed00 P 2cf438fb48b74762a1bb3d92b1fa1c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aed906dd8144f498fde51977e480b38 in term 0.
I20260927 18:41:33.780242 25180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f038b840f67c4103a8cd36bbde70ed00" candidate_uuid: "7aed906dd8144f498fde51977e480b38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cf438fb48b74762a1bb3d92b1fa1c6e"
I20260927 18:41:33.780313 25180 raft_consensus.cc:3037] T f038b840f67c4103a8cd36bbde70ed00 P 2cf438fb48b74762a1bb3d92b1fa1c6e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:33.781314 25180 raft_consensus.cc:2454] T f038b840f67c4103a8cd36bbde70ed00 P 2cf438fb48b74762a1bb3d92b1fa1c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aed906dd8144f498fde51977e480b38 in term 1.
I20260927 18:41:33.802515 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:33.798857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36414 (local address 127.11.251.194:37093)
0927 18:41:33.798913 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:33.798915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:33.798923 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:41:33.798976 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:33.798977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:33.799009 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:33.799080 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:41:33.799310 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.800135 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.801249 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:33.801545 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:33.801674 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:33.801882 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:41:33.802048 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:41:33.802134 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:33.802401 (+ 267us) server_negotiation.cc:299] Negotiation successful
0927 18:41:33.802426 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260927 18:41:33.805281 25180 raft_consensus.cc:1261] T f038b840f67c4103a8cd36bbde70ed00 P 2cf438fb48b74762a1bb3d92b1fa1c6e [term 1 FOLLOWER]: Refusing update from remote peer 7aed906dd8144f498fde51977e480b38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:33.805281 25445 raft_consensus.cc:1261] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff [term 1 FOLLOWER]: Refusing update from remote peer 7aed906dd8144f498fde51977e480b38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:33.805830 25542 consensus_queue.cc:1035] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a2cde3b809c42a7828ad330df767eff" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 43679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:33.805985 25537 consensus_queue.cc:1035] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cf438fb48b74762a1bb3d92b1fa1c6e" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:33.809351 25545 mvcc.cc:205] Tried to move back new op lower bound from 7334029286622740480 to 7334029286497054720. Current Snapshot: MvccSnapshot[applied={T|T < 7334029286622740480}]
I20260927 18:41:33.810704 25547 mvcc.cc:205] Tried to move back new op lower bound from 7334029286622740480 to 7334029286497054720. Current Snapshot: MvccSnapshot[applied={T|T < 7334029286622740480}]
I20260927 18:41:33.813866 25028 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:41:33.813987 25028 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:41:33.815250 25028 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e6fb6bd973f4c019e9453cdafd71f37: Sending DeleteTablet for 3 replicas of tablet f038b840f67c4103a8cd36bbde70ed00
I20260927 18:41:33.815725 25429 tablet_service.cc:1514] Processing DeleteTablet for tablet f038b840f67c4103a8cd36bbde70ed00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:33 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:56320
I20260927 18:41:33.816130 25161 tablet_service.cc:1514] Processing DeleteTablet for tablet f038b840f67c4103a8cd36bbde70ed00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:33 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:48736
I20260927 18:41:33.816542 25295 tablet_service.cc:1514] Processing DeleteTablet for tablet f038b840f67c4103a8cd36bbde70ed00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:33 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:36406
I20260927 18:41:33.816627 25546 mvcc.cc:205] Tried to move back new op lower bound from 7334029286622740480 to 7334029286497054720. Current Snapshot: MvccSnapshot[applied={T|T < 7334029286622740480}]
I20260927 18:41:33.817754 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 25067
I20260927 18:41:33.822772 25556 tablet_replica.cc:330] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38: stopping tablet replica
I20260927 18:41:33.822976 25554 tablet_replica.cc:330] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff: stopping tablet replica
I20260927 18:41:33.823243 25554 raft_consensus.cc:2229] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:41:33.823436 25556 raft_consensus.cc:2229] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:41:33.823649 25556 raft_consensus.cc:2258] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:41:33.823849 25554 raft_consensus.cc:2258] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:41:33.825289 25556 ts_tablet_manager.cc:1905] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:41:33.831404 25554 ts_tablet_manager.cc:1905] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:41:33.833427 25556 ts_tablet_manager.cc:1918] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:41:33.833662 25554 ts_tablet_manager.cc:1918] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:41:33.833814 25554 log.cc:1196] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/f038b840f67c4103a8cd36bbde70ed00
I20260927 18:41:33.833952 25556 log.cc:1196] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/f038b840f67c4103a8cd36bbde70ed00
I20260927 18:41:33.834328 25554 ts_tablet_manager.cc:1939] T f038b840f67c4103a8cd36bbde70ed00 P 1a2cde3b809c42a7828ad330df767eff: Deleting consensus metadata
I20260927 18:41:33.834645 25556 ts_tablet_manager.cc:1939] T f038b840f67c4103a8cd36bbde70ed00 P 7aed906dd8144f498fde51977e480b38: Deleting consensus metadata
I20260927 18:41:33.842231 25013 catalog_manager.cc:4841] TS 7aed906dd8144f498fde51977e480b38 (127.11.251.194:37093): tablet f038b840f67c4103a8cd36bbde70ed00 (table test-table [id=293754cd037f4d4dae4a27b700e1c3ef]) successfully deleted
I20260927 18:41:33.842948 25016 catalog_manager.cc:4841] TS 1a2cde3b809c42a7828ad330df767eff (127.11.251.195:43679): tablet f038b840f67c4103a8cd36bbde70ed00 (table test-table [id=293754cd037f4d4dae4a27b700e1c3ef]) successfully deleted
W20260927 18:41:33.845496 25014 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.251.193:39603 (error 108)
I20260927 18:41:33.849203 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 25229
I20260927 18:41:33.853711 25523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:33.853499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:39603 (local address 127.0.0.1:48748)
0927 18:41:33.853569 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:33.853662 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.193:39603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260927 18:41:33.853910 25014 catalog_manager.cc:4584] TS 2cf438fb48b74762a1bb3d92b1fa1c6e (127.11.251.193:39603): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f038b840f67c4103a8cd36bbde70ed00: Network error: Client connection negotiation failed: client connection to 127.11.251.193:39603: connect: Connection refused (error 111)
I20260927 18:41:33.859438 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 25364
I20260927 18:41:33.866883 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 24996
I20260927 18:41:33.876034 12271 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260927 18:41:34.220824 12271 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260927 18:41:34.220949 12271 mini_ranger.cc:66] Stopping Ranger...
I20260927 18:41:34.565521 12271 mini_ranger.cc:68] Stopped Ranger
2026-09-27 18:41:34.565 UTC [24063] LOG: received smart shutdown request
2026-09-27 18:41:34.569 UTC [24063] LOG: background worker "logical replication launcher" (PID 24072) exited with exit code 1
2026-09-27 18:41:34.570 UTC [24067] LOG: shutting down
2026-09-27 18:41:34.762 UTC [24063] LOG: database system is shut down
2026-09-27T18:41:34Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52281 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:34 dist-test-slave-z47j krb5kdc[25566](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:34 dist-test-slave-z47j krb5kdc[25566](info): set up 2 sockets
Sep 27 18:41:34 dist-test-slave-z47j krb5kdc[25566](info): commencing operation
krb5kdc: starting...
W20260927 18:41:35.953884 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:41:36 dist-test-slave-z47j krb5kdc[25566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:41:36.101070 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:44305
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:38435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:44305,127.11.251.253:37129,127.11.251.252:37699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:36.193670 25579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:36.193874 25579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:36.193895 25579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:36.195073 25579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:36.195119 25579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:36.195133 25579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:36.195147 25579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:36.195158 25579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:36.196640 25579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:38435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:44305,127.11.251.253:37129,127.11.251.252:37699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:44305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:36.196877 25579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:36.197131 25579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:36.202039 25584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:36.202077 25585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:36.202041 25587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:36.202539 25579 server_base.cc:1034] running on GCE node
I20260927 18:41:36.202913 25579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:36.203203 25579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:36.204380 25579 hybrid_clock.cc:648] HybridClock initialized: now 1790534496204350 us; error 50 us; skew 500 ppm
Sep 27 18:41:36 dist-test-slave-z47j krb5kdc[25566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534496, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:36.207085 25579 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:41:36.207566 25579 webserver.cc:460] Webserver started at http://127.11.251.254:40611/ using document root <none> and password file <none>
I20260927 18:41:36.207810 25579 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:36.207880 25579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:36.208052 25579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:36.209108 25579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "b9054662fdfd47e0a5b7601c64e697d5"
format_stamp: "Formatted at 2026-09-27 18:41:36 on dist-test-slave-z47j"
server_key: "2f8ed6f61b9f42bf6a6bc3f24a709c1f"
server_key_iv: "b0a59cc9d0f414b791d71abe707b111f"
server_key_version: "encryptionkey@0"
I20260927 18:41:36.209467 25579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "b9054662fdfd47e0a5b7601c64e697d5"
format_stamp: "Formatted at 2026-09-27 18:41:36 on dist-test-slave-z47j"
server_key: "2f8ed6f61b9f42bf6a6bc3f24a709c1f"
server_key_iv: "b0a59cc9d0f414b791d71abe707b111f"
server_key_version: "encryptionkey@0"
I20260927 18:41:36.210858 25579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:41:36.211673 25594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:36.211901 25579 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:41:36.212000 25579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "b9054662fdfd47e0a5b7601c64e697d5"
format_stamp: "Formatted at 2026-09-27 18:41:36 on dist-test-slave-z47j"
server_key: "2f8ed6f61b9f42bf6a6bc3f24a709c1f"
server_key_iv: "b0a59cc9d0f414b791d71abe707b111f"
server_key_version: "encryptionkey@0"
I20260927 18:41:36.212090 25579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:36.226862 25579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:36.228754 25579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:36.228910 25579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:36.233623 25579 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:44305
I20260927 18:41:36.233731 25646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:44305 every 8 connection(s)
I20260927 18:41:36.234035 25579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:36.235042 25647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:36.236572 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 25579
I20260927 18:41:36.236721 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:41:36.236953 12271 external_mini_cluster.cc:1428] Setting key 05a4fcdc31b568954041e9d8605ab635
I20260927 18:41:36.237084 25647 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:41:36 dist-test-slave-z47j krb5kdc[25566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534496, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.251.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.253@KRBTEST.COM" created.
I20260927 18:41:36.248519 25650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.238056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47216 (local address 127.11.251.254:44305)
0927 18:41:36.238228 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.238238 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.238401 (+ 163us) server_negotiation.cc:407] Connection header received
0927 18:41:36.239303 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.239306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.239328 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.239456 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:36.240073 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.240948 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.242046 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.242380 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.242519 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.244756 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:36.244764 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:36.244764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:36.244771 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:36.246529 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.247035 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.247038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.247038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.247122 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.247399 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.247400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.247400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.247790 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:36.247892 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.248202 (+ 310us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.248330 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":65,"threads_started":1}
I20260927 18:41:36.248704 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.237976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:44305 (local address 127.0.0.1:47216)
0927 18:41:36.238223 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.238272 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:36.239117 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:36.239462 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:36.239464 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:36.239785 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:36.239962 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.239966 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.241065 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.241066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.241953 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.241955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.242487 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.242489 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.242553 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.243161 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:36.243170 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:36.244654 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:36.246656 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:36.246663 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:36.246666 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:36.246942 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:36.247191 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:36.247191 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:36.247192 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:36.247300 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:36.247915 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:36.247924 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:36.248026 (+ 102us) client_negotiation.cc:769] Sending connection context
0927 18:41:36.248169 (+ 143us) client_negotiation.cc:240] Negotiation successful
0927 18:41:36.248465 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":87,"thread_start_us":98,"threads_started":1}
I20260927 18:41:36.249787 25647 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:41:36.250276 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.250125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:37129 (local address 127.0.0.1:41814)
0927 18:41:36.250195 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.250257 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.253:37129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260927 18:41:36.250422 25596 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.251.253:37129: connect: Connection refused (error 111)
I20260927 18:41:36.250882 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.250798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:37129 (local address 127.0.0.1:41822)
0927 18:41:36.250830 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.250857 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.253:37129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260927 18:41:36.251021 25647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.253:37129: Network error: Client connection negotiation failed: client connection to 127.11.251.253:37129: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.253.keytab.
Entry for principal kudu/127.11.251.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.253.keytab.
I20260927 18:41:36.264780 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.253:37129
--webserver_interface=127.11.251.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.253.keytab
--principal=kudu/127.11.251.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:38435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:44305,127.11.251.253:37129,127.11.251.252:37699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:41:36.300278 25647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } attempt: 1
I20260927 18:41:36.300910 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.300749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:37129 (local address 127.0.0.1:41824)
0927 18:41:36.300839 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.300878 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.253:37129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260927 18:41:36.301286 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.301193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:37129 (local address 127.0.0.1:41840)
0927 18:41:36.301237 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.301263 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.253:37129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260927 18:41:36.301397 25647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.253:37129: Network error: Client connection negotiation failed: client connection to 127.11.251.253:37129: connect: Connection refused (error 111)
W20260927 18:41:36.346125 25654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:36.346364 25654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:36.346400 25654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:36.347513 25654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:36.347584 25654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:36.347609 25654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:36.347631 25654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:36.347661 25654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:36.349138 25654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:38435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:44305,127.11.251.253:37129,127.11.251.252:37699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.253.keytab
--principal=kudu/127.11.251.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.253:37129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.251.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:36.349421 25654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:36.349701 25654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:36.354673 25659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:36.354671 25662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:36.354993 25660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:36.355195 25654 server_base.cc:1034] running on GCE node
I20260927 18:41:36.355531 25654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:36.355995 25654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:36.357213 25654 hybrid_clock.cc:648] HybridClock initialized: now 1790534496357183 us; error 52 us; skew 500 ppm
Sep 27 18:41:36 dist-test-slave-z47j krb5kdc[25566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534496, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:36.360236 25654 init.cc:377] Logged in from keytab as kudu/127.11.251.253@KRBTEST.COM (short username kudu)
I20260927 18:41:36.360714 25654 webserver.cc:460] Webserver started at http://127.11.251.253:37431/ using document root <none> and password file <none>
I20260927 18:41:36.361001 25654 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:36.361076 25654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:36.361238 25654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:36.362507 25654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-1/data/instance:
uuid: "68ae537b4f7b4b0e88f5d1259b272147"
format_stamp: "Formatted at 2026-09-27 18:41:36 on dist-test-slave-z47j"
server_key: "cf349dbe4d143f11fb3fe63bdef16c4d"
server_key_iv: "0b8f2261536857b28775727844a24068"
server_key_version: "encryptionkey@0"
I20260927 18:41:36.362926 25654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-1/wal/instance:
uuid: "68ae537b4f7b4b0e88f5d1259b272147"
format_stamp: "Formatted at 2026-09-27 18:41:36 on dist-test-slave-z47j"
server_key: "cf349dbe4d143f11fb3fe63bdef16c4d"
server_key_iv: "0b8f2261536857b28775727844a24068"
server_key_version: "encryptionkey@0"
I20260927 18:41:36.364444 25654 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 18:41:36.365340 25669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:36.365622 25654 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:41:36.365742 25654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-1/wal
uuid: "68ae537b4f7b4b0e88f5d1259b272147"
format_stamp: "Formatted at 2026-09-27 18:41:36 on dist-test-slave-z47j"
server_key: "cf349dbe4d143f11fb3fe63bdef16c4d"
server_key_iv: "0b8f2261536857b28775727844a24068"
server_key_version: "encryptionkey@0"
I20260927 18:41:36.365836 25654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:36.369551 25647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } attempt: 2
I20260927 18:41:36.370097 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.369941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:37129 (local address 127.0.0.1:41856)
0927 18:41:36.370002 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.370053 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.253:37129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260927 18:41:36.370565 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.370476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:37129 (local address 127.0.0.1:41864)
0927 18:41:36.370524 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.370550 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.253:37129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260927 18:41:36.370663 25647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.253:37129: Network error: Client connection negotiation failed: client connection to 127.11.251.253:37129: connect: Connection refused (error 111)
I20260927 18:41:36.378818 25654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:36.381749 25654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:36.381886 25654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:36.387581 25654 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.253:37129
I20260927 18:41:36.387650 25721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.253:37129 every 8 connection(s)
I20260927 18:41:36.388104 25654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-1/data/info.pb
I20260927 18:41:36.389191 25722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:36.390326 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 25654
I20260927 18:41:36.390475 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-1/wal/instance
I20260927 18:41:36.390724 12271 external_mini_cluster.cc:1428] Setting key e51eb794673e153bd115cc11f4db4667
I20260927 18:41:36.391098 25722 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:41:36 dist-test-slave-z47j krb5kdc[25566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534496, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.253@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.251.252@KRBTEST.COM; defaulting to no policy
I20260927 18:41:36.401113 25650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.391959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47220 (local address 127.11.251.254:44305)
0927 18:41:36.392028 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.392030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.392330 (+ 300us) server_negotiation.cc:407] Connection header received
0927 18:41:36.392832 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.392834 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.392849 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.392912 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:36.393557 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.394080 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.395087 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.395293 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.395367 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.398032 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:36.398034 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:36.398034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:36.398041 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:36.399716 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.400132 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.400134 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.400134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.400191 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.400446 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.400446 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.400447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.400586 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:36.400668 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.400938 (+ 270us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.401018 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:41:36.401309 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.391978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:44305 (local address 127.0.0.1:47220)
0927 18:41:36.392186 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.392225 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:36.392721 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:36.392987 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:36.392991 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:36.393264 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:36.393458 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.393462 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.394198 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.394199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.394981 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.394983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.395385 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.395386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.395515 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.396197 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:36.396205 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:36.397936 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:36.399809 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:36.399816 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:36.399818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:36.400035 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:36.400262 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:36.400263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:36.400263 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:36.400369 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:36.400660 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:36.400667 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:36.400806 (+ 139us) client_negotiation.cc:769] Sending connection context
0927 18:41:36.400917 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 18:41:36.401070 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":102,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.252@KRBTEST.COM" created.
I20260927 18:41:36.402182 25722 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:41:36 dist-test-slave-z47j krb5kdc[25566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534496, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.253@KRBTEST.COM for kudu/127.11.251.253@KRBTEST.COM
I20260927 18:41:36.410122 25726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.402543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41866 (local address 127.11.251.253:37129)
0927 18:41:36.402717 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.402722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.402744 (+ 22us) server_negotiation.cc:407] Connection header received
0927 18:41:36.402776 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.402778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.402789 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.402858 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:36.403312 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.404000 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.404723 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.405029 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.405130 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.407085 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:36.407096 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:36.407096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:36.407104 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:36.408798 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.409177 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.409184 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.409185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.409234 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.409429 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.409430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.409431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.409665 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:36.409765 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.409875 (+ 110us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.409974 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":112,"threads_started":1}
I20260927 18:41:36.410125 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.402451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:37129 (local address 127.0.0.1:41866)
0927 18:41:36.402519 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.402532 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:36.402582 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:36.402923 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:36.402924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:36.403122 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:36.403222 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.403224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.404121 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.404122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.404625 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.404627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.405131 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.405131 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.405198 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.405601 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:36.405606 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:36.406987 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:36.408904 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:36.408906 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:36.408906 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:36.409082 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:36.409301 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:36.409302 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:36.409302 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:36.409358 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:36.409736 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:36.409737 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:36.409779 (+ 42us) client_negotiation.cc:769] Sending connection context
0927 18:41:36.409845 (+ 66us) client_negotiation.cc:240] Negotiation successful
0927 18:41:36.410037 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":84}
I20260927 18:41:36.411196 25722 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:41:36.411598 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.411473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:37699 (local address 127.0.0.1:41228)
0927 18:41:36.411529 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.411577 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260927 18:41:36.411757 25673 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.251.252:37699: connect: Connection refused (error 111)
I20260927 18:41:36.412243 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.412135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:37699 (local address 127.0.0.1:41238)
0927 18:41:36.412188 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.412216 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260927 18:41:36.412348 25722 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.252:37699: Network error: Client connection negotiation failed: client connection to 127.11.251.252:37699: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.252.keytab.
Entry for principal kudu/127.11.251.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.252.keytab.
I20260927 18:41:36.420646 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.252:37699
--webserver_interface=127.11.251.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.252.keytab
--principal=kudu/127.11.251.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:38435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:44305,127.11.251.253:37129,127.11.251.252:37699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:41:36.461673 25722 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } attempt: 1
I20260927 18:41:36.461824 25647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } attempt: 3
I20260927 18:41:36.462391 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.462190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:37699 (local address 127.0.0.1:41244)
0927 18:41:36.462301 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.462343 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260927 18:41:36.462870 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.462700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:37699 (local address 127.0.0.1:41250)
0927 18:41:36.462767 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.462816 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260927 18:41:36.463054 25722 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.252:37699: Network error: Client connection negotiation failed: client connection to 127.11.251.252:37699: connect: Connection refused (error 111)
Sep 27 18:41:36 dist-test-slave-z47j krb5kdc[25566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534496, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for kudu/127.11.251.253@KRBTEST.COM
I20260927 18:41:36.469859 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.462209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:37129 (local address 127.0.0.1:41874)
0927 18:41:36.462281 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.462298 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:36.462352 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:36.462625 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:36.462626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:36.462853 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:36.462978 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.462981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.463811 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.463812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.464370 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.464372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.464806 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.464808 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.464937 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.465495 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:36.465501 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:36.466998 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:36.468783 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:36.468785 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:36.468785 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:36.468968 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:36.469168 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:36.469169 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:36.469169 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:36.469236 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:36.469511 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:36.469513 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:36.469563 (+ 50us) client_negotiation.cc:769] Sending connection context
0927 18:41:36.469647 (+ 84us) client_negotiation.cc:240] Negotiation successful
0927 18:41:36.469754 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260927 18:41:36.469856 25726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.462325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41874 (local address 127.11.251.253:37129)
0927 18:41:36.462385 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.462388 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.462402 (+ 14us) server_negotiation.cc:407] Connection header received
0927 18:41:36.462437 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.462438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.462449 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.462601 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:36.463108 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.463679 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.464457 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.464694 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.464770 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.467111 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:36.467112 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:36.467113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:36.467121 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:36.468676 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.469042 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.469043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.469044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.469101 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.469301 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.469302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.469303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.469435 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:36.469511 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.469670 (+ 159us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.469754 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:41:36.470300 25647 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:41:36.470697 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.470595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:37699 (local address 127.0.0.1:41266)
0927 18:41:36.470646 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.470675 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260927 18:41:36.471069 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.470980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:37699 (local address 127.0.0.1:41274)
0927 18:41:36.471027 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.471050 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260927 18:41:36.471158 25647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.252:37699: Network error: Client connection negotiation failed: client connection to 127.11.251.252:37699: connect: Connection refused (error 111)
I20260927 18:41:36.502307 25647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } attempt: 1
I20260927 18:41:36.502880 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.502733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:37699 (local address 127.0.0.1:41290)
0927 18:41:36.502803 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.502847 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260927 18:41:36.503402 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.503205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:37699 (local address 127.0.0.1:41298)
0927 18:41:36.503294 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.503353 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260927 18:41:36.503592 25647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.252:37699: Network error: Client connection negotiation failed: client connection to 127.11.251.252:37699: connect: Connection refused (error 111)
W20260927 18:41:36.503872 25729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:36.504061 25729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:36.504110 25729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:36.505307 25729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:36.505352 25729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:36.505365 25729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:36.505378 25729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:36.505391 25729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:36.506935 25729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:38435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:44305,127.11.251.253:37129,127.11.251.252:37699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.252.keytab
--principal=kudu/127.11.251.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.252:37699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.251.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:36.507200 25729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:36.507447 25729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:36.512655 25735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:36.512669 25734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:36.512782 25729 server_base.cc:1034] running on GCE node
W20260927 18:41:36.512648 25737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:36.513278 25729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:36.513716 25729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:36.514880 25729 hybrid_clock.cc:648] HybridClock initialized: now 1790534496514855 us; error 62 us; skew 500 ppm
Sep 27 18:41:36 dist-test-slave-z47j krb5kdc[25566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534496, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:36.517859 25729 init.cc:377] Logged in from keytab as kudu/127.11.251.252@KRBTEST.COM (short username kudu)
I20260927 18:41:36.518395 25729 webserver.cc:460] Webserver started at http://127.11.251.252:45031/ using document root <none> and password file <none>
I20260927 18:41:36.518654 25729 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:36.518718 25729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:36.518870 25729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:36.519984 25729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-2/data/instance:
uuid: "53f7b05301c144689716c26ae2e7426f"
format_stamp: "Formatted at 2026-09-27 18:41:36 on dist-test-slave-z47j"
server_key: "4cc3a17f5d65daf7cc13184f026eea3f"
server_key_iv: "3cf1e7bf5dc861cd3336e1ead934e316"
server_key_version: "encryptionkey@0"
I20260927 18:41:36.520363 25729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-2/wal/instance:
uuid: "53f7b05301c144689716c26ae2e7426f"
format_stamp: "Formatted at 2026-09-27 18:41:36 on dist-test-slave-z47j"
server_key: "4cc3a17f5d65daf7cc13184f026eea3f"
server_key_iv: "3cf1e7bf5dc861cd3336e1ead934e316"
server_key_version: "encryptionkey@0"
I20260927 18:41:36.521857 25729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 18:41:36.522732 25744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:36.522987 25729 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:41:36.523092 25729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-2/wal
uuid: "53f7b05301c144689716c26ae2e7426f"
format_stamp: "Formatted at 2026-09-27 18:41:36 on dist-test-slave-z47j"
server_key: "4cc3a17f5d65daf7cc13184f026eea3f"
server_key_iv: "3cf1e7bf5dc861cd3336e1ead934e316"
server_key_version: "encryptionkey@0"
I20260927 18:41:36.523176 25729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:36.531205 25722 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } attempt: 2
I20260927 18:41:36.531790 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.531632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:37699 (local address 127.0.0.1:41300)
0927 18:41:36.531708 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.531759 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260927 18:41:36.532303 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.532211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:37699 (local address 127.0.0.1:41302)
0927 18:41:36.532261 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.532287 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260927 18:41:36.532411 25722 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.252:37699: Network error: Client connection negotiation failed: client connection to 127.11.251.252:37699: connect: Connection refused (error 111)
I20260927 18:41:36.554916 25729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:36.556931 25729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:36.557049 25729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:36.561532 25729 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.252:37699
I20260927 18:41:36.561689 25796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.252:37699 every 8 connection(s)
I20260927 18:41:36.561990 25729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-2/data/info.pb
I20260927 18:41:36.563060 25797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:36.564899 25797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:41:36.566555 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 25729
I20260927 18:41:36.566692 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534329098323-12271-0/minicluster-data/master-2/wal/instance
I20260927 18:41:36.566959 12271 external_mini_cluster.cc:1428] Setting key 66e98b55774ff0dde63932652844c015
Sep 27 18:41:36 dist-test-slave-z47j krb5kdc[25566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534496, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.252@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
Sep 27 18:41:36 dist-test-slave-z47j krb5kdc[25566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.252@KRBTEST.COM
I20260927 18:41:36.576423 25650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.565713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47230 (local address 127.11.251.254:44305)
0927 18:41:36.565796 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.565799 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.566065 (+ 266us) server_negotiation.cc:407] Connection header received
0927 18:41:36.566772 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.566774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.566793 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.566872 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:36.567889 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.568393 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.569922 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.570115 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.570204 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.573020 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:36.573023 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:36.573023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:36.573031 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:36.574779 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.575320 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.575321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.575321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.575373 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.575686 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.575687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.575687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.575803 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:36.575890 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.576202 (+ 312us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.576296 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:41:36.576637 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.565687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:44305 (local address 127.0.0.1:47230)
0927 18:41:36.565919 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.565961 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:36.566642 (+ 681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:36.566965 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:36.566969 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:36.567401 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:36.567760 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.567766 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.568530 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.568530 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.569823 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.569826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.570217 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.570219 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.570377 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.570987 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:36.570998 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:36.572912 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:36.574869 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:36.574878 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:36.574881 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:36.575235 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:36.575468 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:36.575469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:36.575470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:36.575607 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:36.575904 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:36.575912 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:36.576057 (+ 145us) client_negotiation.cc:769] Sending connection context
0927 18:41:36.576179 (+ 122us) client_negotiation.cc:240] Negotiation successful
0927 18:41:36.576362 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":17,"thread_start_us":120,"threads_started":1}
I20260927 18:41:36.577253 25797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:41:36.577373 25801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.567443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41312 (local address 127.11.251.252:37699)
0927 18:41:36.567631 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.567635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.567651 (+ 16us) server_negotiation.cc:407] Connection header received
0927 18:41:36.567725 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.567727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.567745 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.567819 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:36.568524 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.569840 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.570653 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.570967 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.571040 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.573791 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:36.573798 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:36.573798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:36.573811 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:36.575763 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.576108 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.576111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.576111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.576173 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.576368 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.576369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.576369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.576575 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:36.576738 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.577158 (+ 420us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.577247 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":152,"thread_start_us":97,"threads_started":1}
I20260927 18:41:36.578752 25647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } attempt: 2
Sep 27 18:41:36 dist-test-slave-z47j krb5kdc[25566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534496, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.252@KRBTEST.COM for kudu/127.11.251.253@KRBTEST.COM
Sep 27 18:41:36 dist-test-slave-z47j krb5kdc[25566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534496, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for kudu/127.11.251.252@KRBTEST.COM
Sep 27 18:41:36 dist-test-slave-z47j krb5kdc[25566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:41:36.585016 25726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.577826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41880 (local address 127.11.251.253:37129)
0927 18:41:36.577876 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.577878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.577888 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:41:36.577913 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.577914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.577923 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.577990 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:36.578324 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.579015 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.579683 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.579906 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.579996 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.582155 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:36.582157 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:36.582157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:36.582165 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:36.583674 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.584013 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.584015 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.584015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.584082 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.584345 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.584346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.584346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.584508 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:36.584584 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.584816 (+ 232us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.584910 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 18:41:36.585363 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.577706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:37129 (local address 127.0.0.1:41880)
0927 18:41:36.577768 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.577780 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:36.577825 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:36.578007 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:36.578008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:36.578165 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:36.578233 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.578235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.579081 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.579082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.579585 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.579587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.579994 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.579995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.580072 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.580537 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:36.580543 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:36.582060 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:36.583753 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:36.583755 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:36.583755 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:36.583937 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:36.584166 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:36.584167 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:36.584168 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:36.584238 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:36.584595 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:36.584596 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:36.584652 (+ 56us) client_negotiation.cc:769] Sending connection context
0927 18:41:36.585109 (+ 457us) client_negotiation.cc:240] Negotiation successful
0927 18:41:36.585249 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260927 18:41:36.585847 25797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:41:36.587189 25801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.579198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41326 (local address 127.11.251.252:37699)
0927 18:41:36.579330 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.579332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.579338 (+ 6us) server_negotiation.cc:407] Connection header received
0927 18:41:36.579356 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.579357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.579366 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.579705 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:36.579735 (+ 30us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.580435 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.581149 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.581451 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.581528 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.584049 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:36.584051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:36.584051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:36.584059 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:36.585946 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.586373 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.586374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.586374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.586457 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.586647 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.586647 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.586648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.586783 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:36.586863 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.587025 (+ 162us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.587105 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260927 18:41:36.587196 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.578990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:37699 (local address 127.0.0.1:41326)
0927 18:41:36.579060 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.579073 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:36.579120 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:36.579439 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:36.579440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:36.579608 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:36.579639 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.579640 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.580530 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.580531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.581062 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.581064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.581526 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.581527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.581628 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.582051 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:36.582056 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:36.583941 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:36.586049 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:36.586050 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:36.586050 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:36.586282 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:36.586509 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:36.586510 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:36.586510 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:36.586573 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:36.586856 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:36.586857 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:36.586908 (+ 51us) client_negotiation.cc:769] Sending connection context
0927 18:41:36.586999 (+ 91us) client_negotiation.cc:240] Negotiation successful
0927 18:41:36.587105 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260927 18:41:36.587739 25650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.578572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47232 (local address 127.11.251.254:44305)
0927 18:41:36.578611 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.578613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.578623 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:41:36.578659 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.578660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.578668 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.578721 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:36.579302 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.580017 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.581001 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.581261 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.581329 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.584080 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:36.584081 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:36.584082 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:36.584088 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:36.586069 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.586459 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.586460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.586460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.586522 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.586722 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.586723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.586724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.586863 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:36.587049 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.587557 (+ 508us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.587643 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:41:36.589519 25647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5: Bootstrap starting.
I20260927 18:41:36.590296 25647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:36.590651 25647 log.cc:824] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5: Log is configured to *not* fsync() on all Append() calls
Sep 27 18:41:36 dist-test-slave-z47j krb5kdc[25566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534496, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.252@KRBTEST.COM for kudu/127.11.251.252@KRBTEST.COM
I20260927 18:41:36.591701 25647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5: No bootstrap required, opened a new log
Sep 27 18:41:36 dist-test-slave-z47j krb5kdc[25566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.253@KRBTEST.COM
I20260927 18:41:36.594193 25647 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9054662fdfd47e0a5b7601c64e697d5" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } } peers { permanent_uuid: "68ae537b4f7b4b0e88f5d1259b272147" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } } peers { permanent_uuid: "53f7b05301c144689716c26ae2e7426f" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } }
I20260927 18:41:36.594360 25647 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:36.594422 25647 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9054662fdfd47e0a5b7601c64e697d5, State: Initialized, Role: FOLLOWER
I20260927 18:41:36.594586 25647 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9054662fdfd47e0a5b7601c64e697d5" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } } peers { permanent_uuid: "68ae537b4f7b4b0e88f5d1259b272147" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } } peers { permanent_uuid: "53f7b05301c144689716c26ae2e7426f" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } }
I20260927 18:41:36.594985 25647 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:36.594950 25804 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9054662fdfd47e0a5b7601c64e697d5" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } } peers { permanent_uuid: "68ae537b4f7b4b0e88f5d1259b272147" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } } peers { permanent_uuid: "53f7b05301c144689716c26ae2e7426f" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } } }
I20260927 18:41:36.595088 25804 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:41:36.595266 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.586134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:37699 (local address 127.0.0.1:41334)
0927 18:41:36.586236 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.586246 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:36.586284 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:36.586522 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:36.586522 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:36.586709 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:36.586750 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.586752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.588177 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.588178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.589047 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.589050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.589697 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.589699 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.589767 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.590167 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:36.590172 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:36.591580 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:36.593879 (+ 2299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:36.593880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:36.593880 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:36.594115 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:36.594361 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:36.594362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:36.594362 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:36.594424 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:36.594975 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:36.594976 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:36.595015 (+ 39us) client_negotiation.cc:769] Sending connection context
0927 18:41:36.595062 (+ 47us) client_negotiation.cc:240] Negotiation successful
0927 18:41:36.595171 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260927 18:41:36.595831 25802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.586209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41334 (local address 127.11.251.252:37699)
0927 18:41:36.586354 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.586356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.586362 (+ 6us) server_negotiation.cc:407] Connection header received
0927 18:41:36.586378 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.586379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.586387 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.586480 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:36.586837 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.588073 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.589146 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.589435 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.589510 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.591676 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:36.591678 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:36.591678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:36.591685 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:36.593782 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.594211 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.594212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.594212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.594275 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.594648 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.594649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.594649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.594893 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:36.595510 (+ 617us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.595561 (+ 51us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.595662 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
W20260927 18:41:36.597429 25815 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:41:36.597488 25815 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:41:36.597582 25797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f: Bootstrap starting.
I20260927 18:41:36.598419 25797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:36.598526 25726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.588908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41886 (local address 127.11.251.253:37129)
0927 18:41:36.588949 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.588950 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.588958 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:41:36.589003 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.589004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.589014 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.589067 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:36.589541 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.590558 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.591443 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.591743 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.591774 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.594343 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:36.594344 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:36.594345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:36.594352 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:36.596377 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.596769 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.596770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.596771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.596827 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.597598 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.597599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.597600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.597729 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:36.597888 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.598329 (+ 441us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.598400 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:41:36.598721 25797 log.cc:824] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:36.599834 25797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f: No bootstrap required, opened a new log
I20260927 18:41:36.598559 25579 master_runner.cc:186] Error getting master registration for 127.11.251.253:37129: OK, instance_id { permanent_uuid: "68ae537b4f7b4b0e88f5d1259b272147" instance_seqno: 1790534496385912 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 18:41:36.600785 25579 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 18:41:36.601933 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:41:36.602088 25797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9054662fdfd47e0a5b7601c64e697d5" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } } peers { permanent_uuid: "68ae537b4f7b4b0e88f5d1259b272147" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } } peers { permanent_uuid: "53f7b05301c144689716c26ae2e7426f" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } }
I20260927 18:41:36.602260 25797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:36.602308 25797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53f7b05301c144689716c26ae2e7426f, State: Initialized, Role: FOLLOWER
I20260927 18:41:36.602433 25797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9054662fdfd47e0a5b7601c64e697d5" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } } peers { permanent_uuid: "68ae537b4f7b4b0e88f5d1259b272147" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } } peers { permanent_uuid: "53f7b05301c144689716c26ae2e7426f" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } }
I20260927 18:41:36.602769 25797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:36.602737 25818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9054662fdfd47e0a5b7601c64e697d5" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } } peers { permanent_uuid: "68ae537b4f7b4b0e88f5d1259b272147" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } } peers { permanent_uuid: "53f7b05301c144689716c26ae2e7426f" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } } }
I20260927 18:41:36.602955 25818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f [sys.catalog]: This master's current role is: FOLLOWER
W20260927 18:41:36.606791 25836 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:41:36.606858 25836 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:41:36.606381 25729 master_runner.cc:186] Error getting master registration for 127.11.251.253:37129: OK, instance_id { permanent_uuid: "68ae537b4f7b4b0e88f5d1259b272147" instance_seqno: 1790534496385912 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 18:41:36.608896 25729 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 18:41:36.612304 25802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.603665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41346 (local address 127.11.251.252:37699)
0927 18:41:36.603709 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.603711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.604126 (+ 415us) server_negotiation.cc:407] Connection header received
0927 18:41:36.604185 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.604185 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.604201 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.604254 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:36.605172 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.605980 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.607054 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.607398 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.607483 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.608984 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:36.608985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:36.608986 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:36.608993 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:36.610841 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.611297 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.611298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.611298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.611355 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.611612 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.611613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.611613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.611761 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:36.611864 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.612096 (+ 232us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.612194 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 18:41:36.614524 25726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.603712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41898 (local address 127.11.251.253:37129)
0927 18:41:36.603744 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.603746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.604270 (+ 524us) server_negotiation.cc:407] Connection header received
0927 18:41:36.604361 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.604362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.604372 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.604401 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:36.605272 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.605981 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.609889 (+ 3908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.610205 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.610249 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.611781 (+ 1532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:36.611782 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:36.611783 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:36.611789 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:36.613343 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.613678 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.613679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.613679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.613729 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.613964 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.613965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.613965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.614103 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:36.614167 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.614369 (+ 202us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.614444 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 18:41:36.618088 25650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.603915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47242 (local address 127.11.251.254:44305)
0927 18:41:36.603953 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.603955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.605129 (+ 1174us) server_negotiation.cc:407] Connection header received
0927 18:41:36.609781 (+ 4652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.609782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.609794 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.609826 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:36.610573 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.611375 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.612709 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.613021 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.613063 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.614845 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:36.614847 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:36.614847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:36.614854 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:36.616721 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.617162 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.617163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.617163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.617225 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.617462 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.617463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.617463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.617644 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:36.617751 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.617907 (+ 156us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.617989 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 18:41:36.623554 25722 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } attempt: 3
Sep 27 18:41:36 dist-test-slave-z47j krb5kdc[25566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534496, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.253@KRBTEST.COM for kudu/127.11.251.252@KRBTEST.COM
I20260927 18:41:36.631625 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.623780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:37699 (local address 127.0.0.1:41356)
0927 18:41:36.623868 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.623878 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:36.623914 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:36.624109 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:36.624110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:36.624265 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:36.624302 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.624304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.625245 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.625246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.626041 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.626044 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.626879 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.626880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.626935 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.627368 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:36.627373 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:36.628786 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:36.630614 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:36.630615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:36.630616 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:36.630812 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:36.631022 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:36.631023 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:36.631023 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:36.631077 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:36.631328 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:36.631329 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:36.631368 (+ 39us) client_negotiation.cc:769] Sending connection context
0927 18:41:36.631425 (+ 57us) client_negotiation.cc:240] Negotiation successful
0927 18:41:36.631530 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260927 18:41:36.631956 25802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.623925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41356 (local address 127.11.251.252:37699)
0927 18:41:36.623975 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.623978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.623987 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:41:36.624008 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.624009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.624018 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.624086 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:36.624395 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.625147 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.626449 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.626761 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.626824 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.628877 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:36.628878 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:36.628878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:36.628885 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:36.630531 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.630883 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.630884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.630885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.630931 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:36.631138 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:36.631138 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:36.631139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:36.631252 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:36.631306 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.631794 (+ 488us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.631866 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 18:41:36.633533 25722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147: Bootstrap starting.
I20260927 18:41:36.634316 25722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:36.634629 25722 log.cc:824] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:36.635637 25722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147: No bootstrap required, opened a new log
I20260927 18:41:36.637705 25722 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9054662fdfd47e0a5b7601c64e697d5" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } } peers { permanent_uuid: "68ae537b4f7b4b0e88f5d1259b272147" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } } peers { permanent_uuid: "53f7b05301c144689716c26ae2e7426f" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } }
I20260927 18:41:36.637847 25722 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:36.637883 25722 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68ae537b4f7b4b0e88f5d1259b272147, State: Initialized, Role: FOLLOWER
I20260927 18:41:36.638033 25722 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9054662fdfd47e0a5b7601c64e697d5" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } } peers { permanent_uuid: "68ae537b4f7b4b0e88f5d1259b272147" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } } peers { permanent_uuid: "53f7b05301c144689716c26ae2e7426f" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } }
I20260927 18:41:36.638357 25722 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:36.638362 25839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9054662fdfd47e0a5b7601c64e697d5" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } } peers { permanent_uuid: "68ae537b4f7b4b0e88f5d1259b272147" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } } peers { permanent_uuid: "53f7b05301c144689716c26ae2e7426f" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } } }
I20260927 18:41:36.638609 25839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [sys.catalog]: This master's current role is: FOLLOWER
W20260927 18:41:36.640982 25850 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:41:36.641084 25850 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:41:36.642745 25654 master_runner.cc:263] No leader master found from master 68ae537b4f7b4b0e88f5d1259b272147
I20260927 18:41:36.642802 25654 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 18:41:36.654747 25839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:41:36.654877 25839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9054662fdfd47e0a5b7601c64e697d5" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } } peers { permanent_uuid: "68ae537b4f7b4b0e88f5d1259b272147" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } } peers { permanent_uuid: "53f7b05301c144689716c26ae2e7426f" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } }
I20260927 18:41:36.655155 25839 leader_election.cc:288] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9054662fdfd47e0a5b7601c64e697d5 (127.11.251.254:44305), 53f7b05301c144689716c26ae2e7426f (127.11.251.252:37699)
I20260927 18:41:36.655426 25621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "68ae537b4f7b4b0e88f5d1259b272147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9054662fdfd47e0a5b7601c64e697d5" is_pre_election: true
I20260927 18:41:36.655601 25621 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68ae537b4f7b4b0e88f5d1259b272147 in term 0.
I20260927 18:41:36.655907 25672 leader_election.cc:302] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68ae537b4f7b4b0e88f5d1259b272147, b9054662fdfd47e0a5b7601c64e697d5; no voters:
I20260927 18:41:36.656049 25839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:41:36.656121 25839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:41:36.656320 25839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:36.656342 25771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "68ae537b4f7b4b0e88f5d1259b272147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53f7b05301c144689716c26ae2e7426f" is_pre_election: true
I20260927 18:41:36.656478 25771 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68ae537b4f7b4b0e88f5d1259b272147 in term 0.
I20260927 18:41:36.657455 25839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9054662fdfd47e0a5b7601c64e697d5" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } } peers { permanent_uuid: "68ae537b4f7b4b0e88f5d1259b272147" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } } peers { permanent_uuid: "53f7b05301c144689716c26ae2e7426f" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } }
I20260927 18:41:36.657647 25839 leader_election.cc:288] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [CANDIDATE]: Term 1 election: Requested vote from peers b9054662fdfd47e0a5b7601c64e697d5 (127.11.251.254:44305), 53f7b05301c144689716c26ae2e7426f (127.11.251.252:37699)
I20260927 18:41:36.657852 25771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "68ae537b4f7b4b0e88f5d1259b272147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53f7b05301c144689716c26ae2e7426f"
I20260927 18:41:36.657959 25771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:36.658972 25771 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ae537b4f7b4b0e88f5d1259b272147 in term 1.
I20260927 18:41:36.659262 25621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "68ae537b4f7b4b0e88f5d1259b272147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9054662fdfd47e0a5b7601c64e697d5"
I20260927 18:41:36.659358 25621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:36.660327 25621 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ae537b4f7b4b0e88f5d1259b272147 in term 1.
I20260927 18:41:36.660522 25673 leader_election.cc:302] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53f7b05301c144689716c26ae2e7426f, 68ae537b4f7b4b0e88f5d1259b272147; no voters:
I20260927 18:41:36.660715 25839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:36.660935 25839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [term 1 LEADER]: Becoming Leader. State: Replica: 68ae537b4f7b4b0e88f5d1259b272147, State: Running, Role: LEADER
I20260927 18:41:36.661046 25839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9054662fdfd47e0a5b7601c64e697d5" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } } peers { permanent_uuid: "68ae537b4f7b4b0e88f5d1259b272147" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } } peers { permanent_uuid: "53f7b05301c144689716c26ae2e7426f" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } }
I20260927 18:41:36.661450 25839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68ae537b4f7b4b0e88f5d1259b272147. Latest consensus state: current_term: 1 leader_uuid: "68ae537b4f7b4b0e88f5d1259b272147" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9054662fdfd47e0a5b7601c64e697d5" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } } peers { permanent_uuid: "68ae537b4f7b4b0e88f5d1259b272147" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } } peers { permanent_uuid: "53f7b05301c144689716c26ae2e7426f" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } } }
I20260927 18:41:36.661531 25839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:36.661787 25854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:36.662022 25854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:36.664333 25771 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f [term 1 FOLLOWER]: Refusing update from remote peer 68ae537b4f7b4b0e88f5d1259b272147: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:36.664692 25839 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53f7b05301c144689716c26ae2e7426f" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:36.666368 25818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68ae537b4f7b4b0e88f5d1259b272147. Latest consensus state: current_term: 1 leader_uuid: "68ae537b4f7b4b0e88f5d1259b272147" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9054662fdfd47e0a5b7601c64e697d5" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } } peers { permanent_uuid: "68ae537b4f7b4b0e88f5d1259b272147" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } } peers { permanent_uuid: "53f7b05301c144689716c26ae2e7426f" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } } }
I20260927 18:41:36.666461 25818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:41:36.666831 25621 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5 [term 1 FOLLOWER]: Refusing update from remote peer 68ae537b4f7b4b0e88f5d1259b272147: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:36.667090 25839 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9054662fdfd47e0a5b7601c64e697d5" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:36.668313 25804 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68ae537b4f7b4b0e88f5d1259b272147. Latest consensus state: current_term: 1 leader_uuid: "68ae537b4f7b4b0e88f5d1259b272147" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9054662fdfd47e0a5b7601c64e697d5" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } } peers { permanent_uuid: "68ae537b4f7b4b0e88f5d1259b272147" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } } peers { permanent_uuid: "53f7b05301c144689716c26ae2e7426f" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } } }
I20260927 18:41:36.668407 25804 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:41:36.669473 25855 mvcc.cc:205] Tried to move back new op lower bound from 7334029298333904896 to 7334029298324226048. Current Snapshot: MvccSnapshot[applied={T|T < 7334029298333904896}]
I20260927 18:41:36.669790 25839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "68ae537b4f7b4b0e88f5d1259b272147" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9054662fdfd47e0a5b7601c64e697d5" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } } peers { permanent_uuid: "68ae537b4f7b4b0e88f5d1259b272147" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } } peers { permanent_uuid: "53f7b05301c144689716c26ae2e7426f" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } } }
I20260927 18:41:36.669891 25839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:36.670243 25804 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "68ae537b4f7b4b0e88f5d1259b272147" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9054662fdfd47e0a5b7601c64e697d5" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } } peers { permanent_uuid: "68ae537b4f7b4b0e88f5d1259b272147" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } } peers { permanent_uuid: "53f7b05301c144689716c26ae2e7426f" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } } }
I20260927 18:41:36.670328 25804 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:41:36.670955 25858 mvcc.cc:205] Tried to move back new op lower bound from 7334029298333904896 to 7334029298324226048. Current Snapshot: MvccSnapshot[applied={T|T < 7334029298333904896}]
I20260927 18:41:36.671784 25839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "68ae537b4f7b4b0e88f5d1259b272147" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9054662fdfd47e0a5b7601c64e697d5" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } } peers { permanent_uuid: "68ae537b4f7b4b0e88f5d1259b272147" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } } peers { permanent_uuid: "53f7b05301c144689716c26ae2e7426f" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } } }
I20260927 18:41:36.671864 25839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:36.672236 25818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "68ae537b4f7b4b0e88f5d1259b272147" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9054662fdfd47e0a5b7601c64e697d5" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 44305 } } peers { permanent_uuid: "68ae537b4f7b4b0e88f5d1259b272147" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 37129 } } peers { permanent_uuid: "53f7b05301c144689716c26ae2e7426f" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 37699 } } }
I20260927 18:41:36.672325 25818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:41:36.672976 25856 mvcc.cc:205] Tried to move back new op lower bound from 7334029298333904896 to 7334029298324226048. Current Snapshot: MvccSnapshot[applied={T|T < 7334029298333904896}]
I20260927 18:41:36.674497 25854 catalog_manager.cc:1347] Generated new cluster ID: 65286e88771a4c6c8f196159b932c8ba
I20260927 18:41:36.674553 25854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:36.705554 25854 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:36.706375 25854 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:36.731905 25854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68ae537b4f7b4b0e88f5d1259b272147: Generated new TSK 0
I20260927 18:41:36.732157 25854 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:41:37.598071 25815 catalog_manager.cc:1259] Loaded cluster ID: 65286e88771a4c6c8f196159b932c8ba
I20260927 18:41:37.598177 25815 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5: loading cluster ID for follower catalog manager: success
I20260927 18:41:37.599086 25815 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5: acquiring CA information for follower catalog manager: success
I20260927 18:41:37.599375 25815 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b9054662fdfd47e0a5b7601c64e697d5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 18:41:37.607409 25836 catalog_manager.cc:1259] Loaded cluster ID: 65286e88771a4c6c8f196159b932c8ba
I20260927 18:41:37.607523 25836 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f: loading cluster ID for follower catalog manager: success
I20260927 18:41:37.608474 25836 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f: acquiring CA information for follower catalog manager: success
I20260927 18:41:37.608883 25836 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 53f7b05301c144689716c26ae2e7426f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 18:41:37.800630 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 25579
I20260927 18:41:37.807729 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 25654
I20260927 18:41:37.813570 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 25729
2026-09-27T18:41:37Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2934 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:37 dist-test-slave-z47j krb5kdc[25871](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:37 dist-test-slave-z47j krb5kdc[25871](info): set up 2 sockets
Sep 27 18:41:37 dist-test-slave-z47j krb5kdc[25871](info): commencing operation
krb5kdc: starting...
W20260927 18:41:38.857858 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:41:38 dist-test-slave-z47j krb5kdc[25871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:41:38.982545 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:46441
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:45263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:46441
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:39.074405 25884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:39.074654 25884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:39.074679 25884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:39.076042 25884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:39.076089 25884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:39.076105 25884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:39.076119 25884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:39.076133 25884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:39.077872 25884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:45263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:46441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:46441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:39.078178 25884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:39.078464 25884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:41:39.084125 25884 server_base.cc:1034] running on GCE node
W20260927 18:41:39.084106 25890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:39.084098 25889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:39.084106 25892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:39.084836 25884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:39.085165 25884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:39.086369 25884 hybrid_clock.cc:648] HybridClock initialized: now 1790534499086364 us; error 52 us; skew 500 ppm
Sep 27 18:41:39 dist-test-slave-z47j krb5kdc[25871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534499, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:39.089318 25884 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:41:39.089877 25884 webserver.cc:460] Webserver started at http://127.11.251.254:42691/ using document root <none> and password file <none>
I20260927 18:41:39.090124 25884 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:39.090194 25884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:39.090355 25884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:39.091571 25884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "91ade7b6a629418fa0f567eaa9962701"
format_stamp: "Formatted at 2026-09-27 18:41:39 on dist-test-slave-z47j"
server_key: "7d1cea47b6f2fb2e170a0e516120649e"
server_key_iv: "bb6b93dbf96591331c640eabb733a626"
server_key_version: "encryptionkey@0"
I20260927 18:41:39.091974 25884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "91ade7b6a629418fa0f567eaa9962701"
format_stamp: "Formatted at 2026-09-27 18:41:39 on dist-test-slave-z47j"
server_key: "7d1cea47b6f2fb2e170a0e516120649e"
server_key_iv: "bb6b93dbf96591331c640eabb733a626"
server_key_version: "encryptionkey@0"
I20260927 18:41:39.093453 25884 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:41:39.094409 25899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:39.094700 25884 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:41:39.094848 25884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "91ade7b6a629418fa0f567eaa9962701"
format_stamp: "Formatted at 2026-09-27 18:41:39 on dist-test-slave-z47j"
server_key: "7d1cea47b6f2fb2e170a0e516120649e"
server_key_iv: "bb6b93dbf96591331c640eabb733a626"
server_key_version: "encryptionkey@0"
I20260927 18:41:39.094937 25884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:39.110070 25884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:39.112236 25884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:39.112413 25884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:39.117215 25884 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:46441
I20260927 18:41:39.117249 25951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:46441 every 8 connection(s)
I20260927 18:41:39.117702 25884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:39.118033 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 25884
I20260927 18:41:39.118157 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:41:39.118360 12271 external_mini_cluster.cc:1428] Setting key 5736c06d9cd8d1043d20247b4b0a4eb4
I20260927 18:41:39.118876 25952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:39.122022 25952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701: Bootstrap starting.
I20260927 18:41:39.122689 25952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:39.123013 25952 log.cc:824] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:39.123924 25952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701: No bootstrap required, opened a new log
Sep 27 18:41:39 dist-test-slave-z47j krb5kdc[25871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:41:39.126336 25952 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91ade7b6a629418fa0f567eaa9962701" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46441 } }
I20260927 18:41:39.126487 25952 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:39.126509 25952 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ade7b6a629418fa0f567eaa9962701, State: Initialized, Role: FOLLOWER
I20260927 18:41:39.126668 25952 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91ade7b6a629418fa0f567eaa9962701" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46441 } }
I20260927 18:41:39.126754 25952 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:39.126798 25952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:39.126864 25952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:39.127434 25952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91ade7b6a629418fa0f567eaa9962701" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46441 } }
I20260927 18:41:39.127558 25952 leader_election.cc:302] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91ade7b6a629418fa0f567eaa9962701; no voters:
I20260927 18:41:39.127774 25952 leader_election.cc:288] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:39.127803 25957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:39.127975 25957 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701 [term 1 LEADER]: Becoming Leader. State: Replica: 91ade7b6a629418fa0f567eaa9962701, State: Running, Role: LEADER
I20260927 18:41:39.128113 25952 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:39.128104 25957 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91ade7b6a629418fa0f567eaa9962701" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46441 } }
I20260927 18:41:39.128530 25957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91ade7b6a629418fa0f567eaa9962701. Latest consensus state: current_term: 1 leader_uuid: "91ade7b6a629418fa0f567eaa9962701" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91ade7b6a629418fa0f567eaa9962701" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46441 } } }
I20260927 18:41:39.128535 25959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91ade7b6a629418fa0f567eaa9962701" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91ade7b6a629418fa0f567eaa9962701" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46441 } } }
I20260927 18:41:39.128698 25957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:39.128804 25959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:39.129845 25955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:39.118971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42848 (local address 127.11.251.254:46441)
0927 18:41:39.119172 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:39.119183 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:39.119218 (+ 35us) server_negotiation.cc:407] Connection header received
0927 18:41:39.119708 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:39.119715 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:39.119763 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:39.119936 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:39.120598 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.121557 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.122409 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.122869 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.123005 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:39.125847 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:39.125853 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:39.125853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:39.125865 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:39.127745 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:39.128173 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:39.128177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:39.128178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:39.128292 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:39.128536 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:39.128537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:39.128537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:39.128938 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:39.129098 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:39.129391 (+ 293us) server_negotiation.cc:299] Negotiation successful
0927 18:41:39.129595 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":96,"threads_started":1}
I20260927 18:41:39.130250 25966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:39.130852 25966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:39.130889 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:41:39.132637 25966 catalog_manager.cc:1347] Generated new cluster ID: 82c7d700115f4f3aa40dda2f61a76081
I20260927 18:41:39.132688 25966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:39.146168 25966 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:39.146732 25966 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:39.155488 25966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91ade7b6a629418fa0f567eaa9962701: Generated new TSK 0
I20260927 18:41:39.155740 25966 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:41:39.185526 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 25884
2026-09-27T18:41:39Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1371 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:39 dist-test-slave-z47j krb5kdc[25979](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:39 dist-test-slave-z47j krb5kdc[25979](info): set up 2 sockets
Sep 27 18:41:39 dist-test-slave-z47j krb5kdc[25979](info): commencing operation
krb5kdc: starting...
W20260927 18:41:40.270133 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:41:40 dist-test-slave-z47j krb5kdc[25979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:41:40.397754 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:42519
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:42519
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:40.486802 25992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:40.487084 25992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:40.487120 25992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:40.488284 25992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:40.488377 25992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:40.488404 25992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:40.488435 25992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:40.488459 25992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:40.490054 25992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:42519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:42519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:40.490346 25992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:40.490602 25992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:40.496121 25997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:40.496111 26000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:40.496147 25992 server_base.cc:1034] running on GCE node
W20260927 18:41:40.496129 25998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:40.496655 25992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:40.497001 25992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:40.498172 25992 hybrid_clock.cc:648] HybridClock initialized: now 1790534500498156 us; error 66 us; skew 500 ppm
Sep 27 18:41:40 dist-test-slave-z47j krb5kdc[25979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534500, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:40.507505 25992 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:41:40.508055 25992 webserver.cc:460] Webserver started at http://127.11.251.254:43115/ using document root <none> and password file <none>
I20260927 18:41:40.508358 25992 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:40.508433 25992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:40.508630 25992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:40.509900 25992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "69a1fe23a8734b86b7b2b98263fe2e7f"
format_stamp: "Formatted at 2026-09-27 18:41:40 on dist-test-slave-z47j"
server_key: "13bc6a178f0709e8ac887d0cca74fc96"
server_key_iv: "df21fe845dabfaaa9cf3f6dfe62a9137"
server_key_version: "encryptionkey@0"
I20260927 18:41:40.510277 25992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "69a1fe23a8734b86b7b2b98263fe2e7f"
format_stamp: "Formatted at 2026-09-27 18:41:40 on dist-test-slave-z47j"
server_key: "13bc6a178f0709e8ac887d0cca74fc96"
server_key_iv: "df21fe845dabfaaa9cf3f6dfe62a9137"
server_key_version: "encryptionkey@0"
I20260927 18:41:40.511773 25992 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:41:40.512583 26007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:40.512789 25992 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:41:40.512902 25992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "69a1fe23a8734b86b7b2b98263fe2e7f"
format_stamp: "Formatted at 2026-09-27 18:41:40 on dist-test-slave-z47j"
server_key: "13bc6a178f0709e8ac887d0cca74fc96"
server_key_iv: "df21fe845dabfaaa9cf3f6dfe62a9137"
server_key_version: "encryptionkey@0"
I20260927 18:41:40.512980 25992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:40.529752 25992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:40.532145 25992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:40.532318 25992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:40.537111 25992 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:42519
I20260927 18:41:40.537184 26059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:42519 every 8 connection(s)
I20260927 18:41:40.537570 25992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:40.538664 26060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:40.541190 26060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f: Bootstrap starting.
I20260927 18:41:40.541950 26060 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:40.542272 26060 log.cc:824] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:40.543012 26060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f: No bootstrap required, opened a new log
I20260927 18:41:40.543614 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 25992
I20260927 18:41:40.543763 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:41:40.544004 12271 external_mini_cluster.cc:1428] Setting key 3996403da52d23c286a25726e05ed6bc
I20260927 18:41:40.545394 26060 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 42519 } }
I20260927 18:41:40.545567 26060 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:40.545650 26060 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69a1fe23a8734b86b7b2b98263fe2e7f, State: Initialized, Role: FOLLOWER
I20260927 18:41:40.545846 26060 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 42519 } }
I20260927 18:41:40.545931 26060 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:40.545972 26060 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:40.546023 26060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:40.546937 26060 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 42519 } }
I20260927 18:41:40.547065 26060 leader_election.cc:302] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69a1fe23a8734b86b7b2b98263fe2e7f; no voters:
I20260927 18:41:40.547248 26060 leader_election.cc:288] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:40.547299 26065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:40.547454 26065 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 1 LEADER]: Becoming Leader. State: Replica: 69a1fe23a8734b86b7b2b98263fe2e7f, State: Running, Role: LEADER
I20260927 18:41:40.547578 26060 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:40.547596 26065 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 42519 } }
I20260927 18:41:40.547996 26065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69a1fe23a8734b86b7b2b98263fe2e7f. Latest consensus state: current_term: 1 leader_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 42519 } } }
I20260927 18:41:40.548074 26065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:40.548233 26066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 42519 } } }
I20260927 18:41:40.548307 26066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:40.548418 26070 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:40.549235 26070 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:40.551445 26070 catalog_manager.cc:1347] Generated new cluster ID: 2f8b7cedff814801aee6d23b2b5583e8
I20260927 18:41:40.551519 26070 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:41:40 dist-test-slave-z47j krb5kdc[25979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:41:40.556392 26064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:40.544667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48748 (local address 127.11.251.254:42519)
0927 18:41:40.544902 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:40.544910 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:40.544945 (+ 35us) server_negotiation.cc:407] Connection header received
0927 18:41:40.545399 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:40.545414 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:40.545471 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:40.545713 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:40.546473 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:40.547652 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:40.548881 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:40.549484 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:40.549986 (+ 502us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:40.552670 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:40.552675 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:40.552675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:40.552687 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:40.554677 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:40.555092 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:40.555095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:40.555095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:40.555223 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:40.555447 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:40.555448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:40.555449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:40.555737 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:40.555897 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:40.556015 (+ 118us) server_negotiation.cc:299] Negotiation successful
0927 18:41:40.556146 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":127,"threads_started":1}
I20260927 18:41:40.565447 26070 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:40.566062 26070 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:40.571054 26070 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f: Generated new TSK 0
I20260927 18:41:40.571255 26070 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.251.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
I20260927 18:41:40.600327 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:42519
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:40.689061 26086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:40.689284 26086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:40.689307 26086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:40.690433 26086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:40.690467 26086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:40.690492 26086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:41:40.691939 26086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:42519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:40.692201 26086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:40.692505 26086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:40.698195 26092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:40.698191 26091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:40.698275 26094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:40.698422 26086 server_base.cc:1034] running on GCE node
I20260927 18:41:40.698657 26086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:40.698918 26086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:40.700088 26086 hybrid_clock.cc:648] HybridClock initialized: now 1790534500700064 us; error 59 us; skew 500 ppm
Sep 27 18:41:40 dist-test-slave-z47j krb5kdc[25979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534500, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:40.702889 26086 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:41:40.703405 26086 webserver.cc:460] Webserver started at http://127.11.251.193:35185/ using document root <none> and password file <none>
I20260927 18:41:40.703648 26086 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:40.703716 26086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:40.703869 26086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:40.704970 26086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "057edf154c3a45a2bfc592ef9e5bce20"
format_stamp: "Formatted at 2026-09-27 18:41:40 on dist-test-slave-z47j"
server_key: "3709558c78db446ea4dacf808da7c95e"
server_key_iv: "5d85c5cbbe6d07dcffa5bca48e0968e5"
server_key_version: "encryptionkey@0"
I20260927 18:41:40.705368 26086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "057edf154c3a45a2bfc592ef9e5bce20"
format_stamp: "Formatted at 2026-09-27 18:41:40 on dist-test-slave-z47j"
server_key: "3709558c78db446ea4dacf808da7c95e"
server_key_iv: "5d85c5cbbe6d07dcffa5bca48e0968e5"
server_key_version: "encryptionkey@0"
I20260927 18:41:40.706820 26086 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:41:40.707595 26101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:40.707796 26086 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:41:40.707895 26086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "057edf154c3a45a2bfc592ef9e5bce20"
format_stamp: "Formatted at 2026-09-27 18:41:40 on dist-test-slave-z47j"
server_key: "3709558c78db446ea4dacf808da7c95e"
server_key_iv: "5d85c5cbbe6d07dcffa5bca48e0968e5"
server_key_version: "encryptionkey@0"
I20260927 18:41:40.707976 26086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:40.720165 26086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:40.722108 26086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:40.722276 26086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:40.722528 26086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:40.722882 26086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:40.722937 26086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:40.722973 26086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:40.723001 26086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:40.729928 26086 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:43337
I20260927 18:41:40.730065 26214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:43337 every 8 connection(s)
I20260927 18:41:40.730367 26086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:41:40.735999 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 26086
I20260927 18:41:40.736145 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:41:40.736404 12271 external_mini_cluster.cc:1428] Setting key 1d237fa652f16e448ef0e5aaa78de374
Sep 27 18:41:40 dist-test-slave-z47j krb5kdc[25979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534500, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:41:40.741456 26217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:40.731154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.193:38525)
0927 18:41:40.731350 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:40.731388 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:40.731981 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:40.732211 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:40.732215 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:40.732834 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:40.733049 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:40.733054 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:40.733815 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:40.733816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:40.734741 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:40.734743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:40.735133 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:40.735134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:40.735245 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:40.735815 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:40.735829 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:40.737871 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:40.739705 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:40.739712 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:40.739715 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:40.740073 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:40.740334 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:40.740335 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:40.740335 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:40.740476 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:40.740857 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:40.740864 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:40.741000 (+ 136us) client_negotiation.cc:769] Sending connection context
0927 18:41:40.741102 (+ 102us) client_negotiation.cc:240] Negotiation successful
0927 18:41:40.741256 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":105,"threads_started":1}
I20260927 18:41:40.741459 26064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:40.731215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:38525 (local address 127.11.251.254:42519)
0927 18:41:40.731309 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:40.731312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:40.731487 (+ 175us) server_negotiation.cc:407] Connection header received
0927 18:41:40.732075 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:40.732077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:40.732094 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:40.732151 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:40.733134 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:40.733725 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:40.734828 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:40.735054 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:40.735111 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:40.737976 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:40.737978 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:40.737979 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:40.737988 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:40.739609 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:40.740176 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:40.740177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:40.740178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:40.740246 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:40.740595 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:40.740597 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:40.740597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:40.740772 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:40.740859 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:40.741154 (+ 295us) server_negotiation.cc:299] Negotiation successful
0927 18:41:40.741303 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 18:41:40.742169 26215 heartbeater.cc:347] Connected to a master server at 127.11.251.254:42519
I20260927 18:41:40.742307 26215 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:40.742514 26215 heartbeater.cc:511] Master 127.11.251.254:42519 requested a full tablet report, sending...
I20260927 18:41:40.743213 26024 ts_manager.cc:194] Registered new tserver with Master: 057edf154c3a45a2bfc592ef9e5bce20 (127.11.251.193:43337)
I20260927 18:41:40.743952 26024 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:38525
WARNING: no policy specified for kudu/127.11.251.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
I20260927 18:41:40.764587 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:42519
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:40.855654 26220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:40.855891 26220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:40.855916 26220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:40.857066 26220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:40.857105 26220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:40.857131 26220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:41:40.858575 26220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:42519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:40.858781 26220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:40.858975 26220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:40.864799 26228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:40.864794 26226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:40.864934 26220 server_base.cc:1034] running on GCE node
W20260927 18:41:40.865010 26225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:40.865291 26220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:40.865552 26220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:40.866757 26220 hybrid_clock.cc:648] HybridClock initialized: now 1790534500866747 us; error 32 us; skew 500 ppm
Sep 27 18:41:40 dist-test-slave-z47j krb5kdc[25979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534500, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:40.869673 26220 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:41:40.870223 26220 webserver.cc:460] Webserver started at http://127.11.251.194:40725/ using document root <none> and password file <none>
I20260927 18:41:40.870468 26220 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:40.870538 26220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:40.870693 26220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:40.871867 26220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "8243ccc2001c4efe8144c247db62a157"
format_stamp: "Formatted at 2026-09-27 18:41:40 on dist-test-slave-z47j"
server_key: "9453a5bf33e1c67688412a08c751e9e6"
server_key_iv: "e94efb69fa4c6ae49f14e2b9a8281543"
server_key_version: "encryptionkey@0"
I20260927 18:41:40.872262 26220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "8243ccc2001c4efe8144c247db62a157"
format_stamp: "Formatted at 2026-09-27 18:41:40 on dist-test-slave-z47j"
server_key: "9453a5bf33e1c67688412a08c751e9e6"
server_key_iv: "e94efb69fa4c6ae49f14e2b9a8281543"
server_key_version: "encryptionkey@0"
I20260927 18:41:40.873705 26220 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 18:41:40.874526 26235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:40.874737 26220 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:41:40.874845 26220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "8243ccc2001c4efe8144c247db62a157"
format_stamp: "Formatted at 2026-09-27 18:41:40 on dist-test-slave-z47j"
server_key: "9453a5bf33e1c67688412a08c751e9e6"
server_key_iv: "e94efb69fa4c6ae49f14e2b9a8281543"
server_key_version: "encryptionkey@0"
I20260927 18:41:40.874930 26220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:40.890450 26220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:40.892635 26220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:40.892813 26220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:40.893085 26220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:40.893427 26220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:40.893482 26220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:40.893524 26220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:40.893567 26220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:40.900523 26220 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:37107
I20260927 18:41:40.900571 26348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:37107 every 8 connection(s)
I20260927 18:41:40.900931 26220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
Sep 27 18:41:40 dist-test-slave-z47j krb5kdc[25979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534500, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:41:40.910383 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 26220
I20260927 18:41:40.910508 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:41:40.910748 12271 external_mini_cluster.cc:1428] Setting key be798f9519cbec5ca26b0022ed7bc3cc
I20260927 18:41:40.910888 26064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:40.901829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:59557 (local address 127.11.251.254:42519)
0927 18:41:40.901908 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:40.901910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:40.902114 (+ 204us) server_negotiation.cc:407] Connection header received
0927 18:41:40.902680 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:40.902681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:40.902693 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:40.902786 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:40.903901 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:40.904460 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:40.905488 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:40.905785 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:40.905846 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:40.908095 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:40.908097 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:40.908097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:40.908103 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:40.909371 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:40.909788 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:40.909789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:40.909790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:40.909844 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:40.910128 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:40.910129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:40.910129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:40.910238 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:40.910291 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:40.910650 (+ 359us) server_negotiation.cc:299] Negotiation successful
0927 18:41:40.910778 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:41:40.911190 26351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:40.901784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.194:59557)
0927 18:41:40.901961 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:40.902007 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:40.902585 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:40.902884 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:40.902887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:40.903503 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:40.903793 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:40.903799 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:40.904541 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:40.904542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:40.905400 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:40.905403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:40.905860 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:40.905861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:40.905970 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:40.906541 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:40.906551 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:40.908006 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:40.909441 (+ 1435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:40.909447 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:40.909449 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:40.909722 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:40.909942 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:40.909943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:40.909944 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:40.910049 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:40.910306 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:40.910314 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:40.910497 (+ 183us) client_negotiation.cc:769] Sending connection context
0927 18:41:40.910623 (+ 126us) client_negotiation.cc:240] Negotiation successful
0927 18:41:40.910904 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260927 18:41:40.911648 26349 heartbeater.cc:347] Connected to a master server at 127.11.251.254:42519
I20260927 18:41:40.911767 26349 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:40.912004 26349 heartbeater.cc:511] Master 127.11.251.254:42519 requested a full tablet report, sending...
I20260927 18:41:40.912408 26023 ts_manager.cc:194] Registered new tserver with Master: 8243ccc2001c4efe8144c247db62a157 (127.11.251.194:37107)
I20260927 18:41:40.912817 26023 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:59557
WARNING: no policy specified for kudu/127.11.251.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
I20260927 18:41:40.937117 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:42519
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:41.031551 26354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:41.031772 26354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:41.031796 26354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:41.032996 26354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:41.033037 26354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:41.033064 26354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:41:41.034621 26354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:42519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:41.034878 26354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:41.035136 26354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:41.040513 26362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:41.040529 26360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:41.040668 26354 server_base.cc:1034] running on GCE node
W20260927 18:41:41.040572 26359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:41.041021 26354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:41.041241 26354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:41.042404 26354 hybrid_clock.cc:648] HybridClock initialized: now 1790534501042385 us; error 37 us; skew 500 ppm
Sep 27 18:41:41 dist-test-slave-z47j krb5kdc[25979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534501, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:41.045295 26354 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:41:41.045867 26354 webserver.cc:460] Webserver started at http://127.11.251.195:45865/ using document root <none> and password file <none>
I20260927 18:41:41.046118 26354 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:41.046192 26354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:41.046351 26354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:41.047552 26354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "e4538361d248498bab4699b9d945e7cc"
format_stamp: "Formatted at 2026-09-27 18:41:41 on dist-test-slave-z47j"
server_key: "f890584ae50388a89d665b9993a4c687"
server_key_iv: "e323d53d0b4869a4ca05c8ae4d82d737"
server_key_version: "encryptionkey@0"
I20260927 18:41:41.048002 26354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "e4538361d248498bab4699b9d945e7cc"
format_stamp: "Formatted at 2026-09-27 18:41:41 on dist-test-slave-z47j"
server_key: "f890584ae50388a89d665b9993a4c687"
server_key_iv: "e323d53d0b4869a4ca05c8ae4d82d737"
server_key_version: "encryptionkey@0"
I20260927 18:41:41.049547 26354 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:41:41.050438 26369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:41.050626 26354 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:41:41.050760 26354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "e4538361d248498bab4699b9d945e7cc"
format_stamp: "Formatted at 2026-09-27 18:41:41 on dist-test-slave-z47j"
server_key: "f890584ae50388a89d665b9993a4c687"
server_key_iv: "e323d53d0b4869a4ca05c8ae4d82d737"
server_key_version: "encryptionkey@0"
I20260927 18:41:41.050849 26354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:41.072427 26354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:41.074990 26354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:41.075166 26354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:41.075430 26354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:41.075785 26354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:41.075829 26354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:41.075855 26354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:41.075868 26354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:41.082954 26354 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:35517
I20260927 18:41:41.083047 26482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:35517 every 8 connection(s)
I20260927 18:41:41.083408 26354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
Sep 27 18:41:41 dist-test-slave-z47j krb5kdc[25979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534501, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:41:41.093050 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 26354
I20260927 18:41:41.093190 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:41:41.093473 12271 external_mini_cluster.cc:1428] Setting key d2ba7260cf29a282b74c71b3b98eecad
I20260927 18:41:41.093988 26064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.084236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:49917 (local address 127.11.251.254:42519)
0927 18:41:41.084304 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:41.084306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:41.084532 (+ 226us) server_negotiation.cc:407] Connection header received
0927 18:41:41.085096 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:41.085097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:41.085109 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:41.085173 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:41.086336 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.086934 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.088004 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.088199 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.088261 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.090702 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:41.090704 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:41.090704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:41.090710 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:41.092276 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:41.092735 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:41.092737 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:41.092737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:41.092803 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:41.093096 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:41.093097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:41.093098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:41.093278 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:41.093348 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:41.093752 (+ 404us) server_negotiation.cc:299] Negotiation successful
0927 18:41:41.093858 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 18:41:41.094264 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.084188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.195:49917)
0927 18:41:41.084379 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:41.084426 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:41.085003 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:41.085272 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:41.085276 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:41.085976 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:41.086214 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.086219 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.087033 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.087035 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.087911 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.087914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.088283 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.088284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.088397 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.088918 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:41.088927 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:41.090588 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:41.092373 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:41.092381 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:41.092384 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:41.092613 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:41.092914 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:41.092915 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:41.092915 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:41.093020 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:41.093352 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:41.093361 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:41.093547 (+ 186us) client_negotiation.cc:769] Sending connection context
0927 18:41:41.093700 (+ 153us) client_negotiation.cc:240] Negotiation successful
0927 18:41:41.093914 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":111,"threads_started":1}
I20260927 18:41:41.094753 26483 heartbeater.cc:347] Connected to a master server at 127.11.251.254:42519
I20260927 18:41:41.094852 26483 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:41.095008 26483 heartbeater.cc:511] Master 127.11.251.254:42519 requested a full tablet report, sending...
I20260927 18:41:41.095417 26023 ts_manager.cc:194] Registered new tserver with Master: e4538361d248498bab4699b9d945e7cc (127.11.251.195:35517)
I20260927 18:41:41.095785 26023 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:49917
I20260927 18:41:41.105533 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:41:41.113869 26064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.107231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48758 (local address 127.11.251.254:42519)
0927 18:41:41.107366 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:41.107369 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:41.107473 (+ 104us) server_negotiation.cc:407] Connection header received
0927 18:41:41.107555 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:41.107556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:41.107568 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:41.107626 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:41.108258 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.108780 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.109533 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.109787 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.109874 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.111186 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:41.111187 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:41.111188 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:41.111193 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:41.112634 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:41.113002 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:41.113003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:41.113004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:41.113055 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:41.113250 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:41.113251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:41.113251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:41.113435 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:41.113527 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:41.113722 (+ 195us) server_negotiation.cc:299] Negotiation successful
0927 18:41:41.113787 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260927 18:41:41.115904 26023 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48758:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:41:41.117879 26023 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:41:41.125190 26497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.120458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:37107 (local address 127.0.0.1:35686)
0927 18:41:41.120919 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:41.120931 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:41.120993 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:41.121595 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:41.121596 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:41.121598 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:41.121637 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.121638 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.122503 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.122504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.123828 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.123831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.124864 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.124865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.124947 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.124962 (+ 15us) client_negotiation.cc:769] Sending connection context
0927 18:41:41.125050 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 18:41:41.125109 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":2,"thread_start_us":92,"threads_started":1}
I20260927 18:41:41.125195 26496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.120584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35686 (local address 127.11.251.194:37107)
0927 18:41:41.121413 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:41.121418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:41.121428 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:41:41.121474 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:41.121476 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:41.121494 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:41.121604 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:41.121701 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.122401 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.123949 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.124775 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.124870 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.124934 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:41.125073 (+ 139us) server_negotiation.cc:299] Negotiation successful
0927 18:41:41.125116 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":778,"thread_start_us":117,"threads_started":1}
I20260927 18:41:41.125517 26494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.120458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:43337 (local address 127.0.0.1:37700)
0927 18:41:41.120737 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:41.120784 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:41.120872 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:41.121238 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:41.121242 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:41.121245 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:41.121340 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.121344 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.122810 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.122811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.123861 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.123863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.125297 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.125298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.125354 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.125358 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:41:41.125410 (+ 52us) client_negotiation.cc:240] Negotiation successful
0927 18:41:41.125453 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":68,"threads_started":1}
I20260927 18:41:41.125844 26498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.120703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37700 (local address 127.11.251.193:43337)
0927 18:41:41.120901 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:41.120907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:41.120924 (+ 17us) server_negotiation.cc:407] Connection header received
0927 18:41:41.120975 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:41.120977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:41.120995 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:41.121111 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:41.121438 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.122682 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.123949 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.124774 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.124870 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.124934 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:41.125692 (+ 758us) server_negotiation.cc:299] Negotiation successful
0927 18:41:41.125747 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":107,"threads_started":1}
I20260927 18:41:41.126104 26499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.120945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40308 (local address 127.11.251.195:35517)
0927 18:41:41.121158 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:41.121162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:41.121210 (+ 48us) server_negotiation.cc:407] Connection header received
0927 18:41:41.121293 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:41.121295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:41.121316 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:41.121422 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:41.121520 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.122701 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.124755 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.125549 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.125678 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.125740 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:41.125932 (+ 192us) server_negotiation.cc:299] Negotiation successful
0927 18:41:41.125990 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":101,"threads_started":1}
I20260927 18:41:41.126526 26283 tablet_service.cc:1467] Processing CreateTablet for tablet 17493cc2e5d04d838a2207753916c3b4 (DEFAULT_TABLE table=test-table [id=f4c63761b1d4454fa11f1d0a11bc52a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:41.126883 26283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17493cc2e5d04d838a2207753916c3b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:41.129935 26500 tablet_bootstrap.cc:492] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157: Bootstrap starting.
I20260927 18:41:41.130158 26149 tablet_service.cc:1467] Processing CreateTablet for tablet 17493cc2e5d04d838a2207753916c3b4 (DEFAULT_TABLE table=test-table [id=f4c63761b1d4454fa11f1d0a11bc52a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:41.130578 26149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17493cc2e5d04d838a2207753916c3b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:41.130796 26500 tablet_bootstrap.cc:654] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:41.131494 26500 log.cc:824] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:41.132486 26500 tablet_bootstrap.cc:492] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157: No bootstrap required, opened a new log
I20260927 18:41:41.132572 26500 ts_tablet_manager.cc:1397] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:41:41.126021 26495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.120669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:35517 (local address 127.0.0.1:40308)
0927 18:41:41.121152 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:41.121162 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:41.121219 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:41.121406 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:41.121407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:41.121410 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:41.121463 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.121465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.123956 (+ 2491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.123956 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.124657 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.124659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.125777 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.125778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.125838 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.125843 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:41:41.125905 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 18:41:41.125952 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":70,"threads_started":1}
I20260927 18:41:41.134742 26502 tablet_bootstrap.cc:492] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20: Bootstrap starting.
I20260927 18:41:41.135043 26500 raft_consensus.cc:348] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } }
I20260927 18:41:41.135264 26500 raft_consensus.cc:374] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:41.135306 26500 raft_consensus.cc:729] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8243ccc2001c4efe8144c247db62a157, State: Initialized, Role: FOLLOWER
I20260927 18:41:41.135500 26502 tablet_bootstrap.cc:654] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:41.135465 26500 consensus_queue.cc:260] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } }
I20260927 18:41:41.135567 26417 tablet_service.cc:1467] Processing CreateTablet for tablet 17493cc2e5d04d838a2207753916c3b4 (DEFAULT_TABLE table=test-table [id=f4c63761b1d4454fa11f1d0a11bc52a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:41.135831 26500 ts_tablet_manager.cc:1428] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260927 18:41:41.135883 26349 heartbeater.cc:503] Master 127.11.251.254:42519 was elected leader, sending a full tablet report...
I20260927 18:41:41.135996 26417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17493cc2e5d04d838a2207753916c3b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:41.136265 26502 log.cc:824] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:41.137480 26502 tablet_bootstrap.cc:492] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20: No bootstrap required, opened a new log
I20260927 18:41:41.137560 26502 ts_tablet_manager.cc:1397] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:41:41.139114 26502 raft_consensus.cc:348] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } }
I20260927 18:41:41.139281 26502 raft_consensus.cc:374] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:41.139299 26505 tablet_bootstrap.cc:492] T 17493cc2e5d04d838a2207753916c3b4 P e4538361d248498bab4699b9d945e7cc: Bootstrap starting.
I20260927 18:41:41.139331 26502 raft_consensus.cc:729] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 057edf154c3a45a2bfc592ef9e5bce20, State: Initialized, Role: FOLLOWER
I20260927 18:41:41.139467 26502 consensus_queue.cc:260] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } }
I20260927 18:41:41.139827 26502 ts_tablet_manager.cc:1428] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:41:41.139873 26215 heartbeater.cc:503] Master 127.11.251.254:42519 was elected leader, sending a full tablet report...
I20260927 18:41:41.140146 26505 tablet_bootstrap.cc:654] T 17493cc2e5d04d838a2207753916c3b4 P e4538361d248498bab4699b9d945e7cc: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:41.140784 26505 log.cc:824] T 17493cc2e5d04d838a2207753916c3b4 P e4538361d248498bab4699b9d945e7cc: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:41.141562 26505 tablet_bootstrap.cc:492] T 17493cc2e5d04d838a2207753916c3b4 P e4538361d248498bab4699b9d945e7cc: No bootstrap required, opened a new log
I20260927 18:41:41.141832 26505 ts_tablet_manager.cc:1397] T 17493cc2e5d04d838a2207753916c3b4 P e4538361d248498bab4699b9d945e7cc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:41:41.143540 26505 raft_consensus.cc:348] T 17493cc2e5d04d838a2207753916c3b4 P e4538361d248498bab4699b9d945e7cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } }
I20260927 18:41:41.143703 26505 raft_consensus.cc:374] T 17493cc2e5d04d838a2207753916c3b4 P e4538361d248498bab4699b9d945e7cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:41.143765 26505 raft_consensus.cc:729] T 17493cc2e5d04d838a2207753916c3b4 P e4538361d248498bab4699b9d945e7cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4538361d248498bab4699b9d945e7cc, State: Initialized, Role: FOLLOWER
I20260927 18:41:41.143893 26505 consensus_queue.cc:260] T 17493cc2e5d04d838a2207753916c3b4 P e4538361d248498bab4699b9d945e7cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } }
I20260927 18:41:41.144179 26505 ts_tablet_manager.cc:1428] T 17493cc2e5d04d838a2207753916c3b4 P e4538361d248498bab4699b9d945e7cc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:41:41.144258 26483 heartbeater.cc:503] Master 127.11.251.254:42519 was elected leader, sending a full tablet report...
W20260927 18:41:41.151520 26350 tablet.cc:2369] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:41:41.231340 26216 tablet.cc:2369] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:41:41.255293 26503 raft_consensus.cc:482] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:41:41.255522 26503 raft_consensus.cc:504] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } }
I20260927 18:41:41.256021 26503 leader_election.cc:288] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 057edf154c3a45a2bfc592ef9e5bce20 (127.11.251.193:43337), e4538361d248498bab4699b9d945e7cc (127.11.251.195:35517)
I20260927 18:41:41.259339 26498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.256437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:40799 (local address 127.11.251.193:43337)
0927 18:41:41.256496 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:41.256498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:41.256513 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:41:41.256592 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:41.256593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:41.256606 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:41.256707 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:41.256989 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.257534 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.258569 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.258945 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.259042 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.259054 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:41.259213 (+ 159us) server_negotiation.cc:299] Negotiation successful
0927 18:41:41.259255 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 18:41:41.259346 26351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.256332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:43337 (local address 127.11.251.194:40799)
0927 18:41:41.256438 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:41.256455 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:41.256515 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:41.256763 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:41.256765 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:41.256767 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:41.256894 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.256897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.257664 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.257665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.258472 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.258475 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.259041 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.259042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.259128 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.259132 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:41:41.259200 (+ 68us) client_negotiation.cc:240] Negotiation successful
0927 18:41:41.259245 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260927 18:41:41.259387 26499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.256444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:47441 (local address 127.11.251.195:35517)
0927 18:41:41.256505 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:41.256507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:41.256740 (+ 233us) server_negotiation.cc:407] Connection header received
0927 18:41:41.256780 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:41.256782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:41.256794 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:41.256872 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:41.257049 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.257547 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.258583 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.259006 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.259094 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.259104 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:41.259252 (+ 148us) server_negotiation.cc:299] Negotiation successful
0927 18:41:41.259303 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:41:41.259444 26509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.256331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:35517 (local address 127.11.251.194:47441)
0927 18:41:41.256625 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:41.256636 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:41.256719 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:41.256881 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:41.256882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:41.256884 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:41.256974 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.256976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.257679 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.257680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.258484 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.258486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.259093 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.259094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.259173 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.259177 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:41:41.259246 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 18:41:41.259288 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":48,"thread_start_us":167,"threads_started":1}
I20260927 18:41:41.260129 26437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17493cc2e5d04d838a2207753916c3b4" candidate_uuid: "8243ccc2001c4efe8144c247db62a157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4538361d248498bab4699b9d945e7cc" is_pre_election: true
I20260927 18:41:41.260215 26169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17493cc2e5d04d838a2207753916c3b4" candidate_uuid: "8243ccc2001c4efe8144c247db62a157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "057edf154c3a45a2bfc592ef9e5bce20" is_pre_election: true
I20260927 18:41:41.260327 26437 raft_consensus.cc:2454] T 17493cc2e5d04d838a2207753916c3b4 P e4538361d248498bab4699b9d945e7cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8243ccc2001c4efe8144c247db62a157 in term 0.
I20260927 18:41:41.260437 26169 raft_consensus.cc:2454] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8243ccc2001c4efe8144c247db62a157 in term 0.
I20260927 18:41:41.260617 26236 leader_election.cc:302] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8243ccc2001c4efe8144c247db62a157, e4538361d248498bab4699b9d945e7cc; no voters:
I20260927 18:41:41.260766 26503 raft_consensus.cc:2781] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:41:41.260845 26503 raft_consensus.cc:482] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:41:41.260874 26503 raft_consensus.cc:3037] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:41.261778 26503 raft_consensus.cc:504] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } }
I20260927 18:41:41.261955 26503 leader_election.cc:288] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 [CANDIDATE]: Term 1 election: Requested vote from peers 057edf154c3a45a2bfc592ef9e5bce20 (127.11.251.193:43337), e4538361d248498bab4699b9d945e7cc (127.11.251.195:35517)
I20260927 18:41:41.262107 26437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17493cc2e5d04d838a2207753916c3b4" candidate_uuid: "8243ccc2001c4efe8144c247db62a157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4538361d248498bab4699b9d945e7cc"
I20260927 18:41:41.262141 26169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17493cc2e5d04d838a2207753916c3b4" candidate_uuid: "8243ccc2001c4efe8144c247db62a157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "057edf154c3a45a2bfc592ef9e5bce20"
I20260927 18:41:41.262204 26437 raft_consensus.cc:3037] T 17493cc2e5d04d838a2207753916c3b4 P e4538361d248498bab4699b9d945e7cc [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:41.262229 26169 raft_consensus.cc:3037] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:41.263017 26437 raft_consensus.cc:2454] T 17493cc2e5d04d838a2207753916c3b4 P e4538361d248498bab4699b9d945e7cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8243ccc2001c4efe8144c247db62a157 in term 1.
I20260927 18:41:41.263068 26169 raft_consensus.cc:2454] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8243ccc2001c4efe8144c247db62a157 in term 1.
I20260927 18:41:41.263221 26236 leader_election.cc:302] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8243ccc2001c4efe8144c247db62a157, e4538361d248498bab4699b9d945e7cc; no voters:
I20260927 18:41:41.263342 26503 raft_consensus.cc:2781] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:41.263495 26503 raft_consensus.cc:686] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 [term 1 LEADER]: Becoming Leader. State: Replica: 8243ccc2001c4efe8144c247db62a157, State: Running, Role: LEADER
I20260927 18:41:41.263600 26503 consensus_queue.cc:237] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } }
I20260927 18:41:41.264647 26023 catalog_manager.cc:5495] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8243ccc2001c4efe8144c247db62a157 (127.11.251.194). New cstate: current_term: 1 leader_uuid: "8243ccc2001c4efe8144c247db62a157" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } health_report { overall_health: UNKNOWN } } }
I20260927 18:41:41.270205 26496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.267682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35700 (local address 127.11.251.194:37107)
0927 18:41:41.267723 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:41.267725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:41.267782 (+ 57us) server_negotiation.cc:407] Connection header received
0927 18:41:41.267863 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:41.267863 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:41.267879 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:41.267936 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:41:41.268050 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.268529 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.269298 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.269486 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.269539 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.269830 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:41:41.269956 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:41:41.270025 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:41.270069 (+ 44us) server_negotiation.cc:299] Negotiation successful
0927 18:41:41.270086 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 18:41:41.272230 26169 raft_consensus.cc:1261] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Refusing update from remote peer 8243ccc2001c4efe8144c247db62a157: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:41.272238 26437 raft_consensus.cc:1261] T 17493cc2e5d04d838a2207753916c3b4 P e4538361d248498bab4699b9d945e7cc [term 1 FOLLOWER]: Refusing update from remote peer 8243ccc2001c4efe8144c247db62a157: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:41.272588 26510 consensus_queue.cc:1035] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:41.272732 26503 consensus_queue.cc:1035] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:41.275804 26515 mvcc.cc:205] Tried to move back new op lower bound from 7334029317208059904 to 7334029317176537088. Current Snapshot: MvccSnapshot[applied={T|T < 7334029317208059904 or (T in {7334029317208059904})}]
I20260927 18:41:41.276646 26514 mvcc.cc:205] Tried to move back new op lower bound from 7334029317208059904 to 7334029317176537088. Current Snapshot: MvccSnapshot[applied={T|T < 7334029317208059904 or (T in {7334029317208059904})}]
I20260927 18:41:41.278656 26023 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:41:41.278796 26023 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:41:41.279703 26023 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f: Sending DeleteTablet for 3 replicas of tablet 17493cc2e5d04d838a2207753916c3b4
I20260927 18:41:41.280095 26283 tablet_service.cc:1514] Processing DeleteTablet for tablet 17493cc2e5d04d838a2207753916c3b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:41 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:35686
I20260927 18:41:41.280126 26149 tablet_service.cc:1514] Processing DeleteTablet for tablet 17493cc2e5d04d838a2207753916c3b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:41 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:37700
I20260927 18:41:41.280236 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 26086
I20260927 18:41:41.280359 26417 tablet_service.cc:1514] Processing DeleteTablet for tablet 17493cc2e5d04d838a2207753916c3b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:41 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:40308
I20260927 18:41:41.281117 26524 tablet_replica.cc:330] T 17493cc2e5d04d838a2207753916c3b4 P e4538361d248498bab4699b9d945e7cc: stopping tablet replica
I20260927 18:41:41.281206 26524 raft_consensus.cc:2229] T 17493cc2e5d04d838a2207753916c3b4 P e4538361d248498bab4699b9d945e7cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:41:41.281291 26524 raft_consensus.cc:2258] T 17493cc2e5d04d838a2207753916c3b4 P e4538361d248498bab4699b9d945e7cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:41:41.281476 26523 tablet_replica.cc:330] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157: stopping tablet replica
I20260927 18:41:41.281566 26523 raft_consensus.cc:2229] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:41:41.281716 26523 raft_consensus.cc:2258] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:41:41.281888 26524 ts_tablet_manager.cc:1905] T 17493cc2e5d04d838a2207753916c3b4 P e4538361d248498bab4699b9d945e7cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:41:41.282164 26523 ts_tablet_manager.cc:1905] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:41:41.283924 26523 ts_tablet_manager.cc:1918] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:41:41.283924 26524 ts_tablet_manager.cc:1918] T 17493cc2e5d04d838a2207753916c3b4 P e4538361d248498bab4699b9d945e7cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:41:41.284008 26524 log.cc:1196] T 17493cc2e5d04d838a2207753916c3b4 P e4538361d248498bab4699b9d945e7cc: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/17493cc2e5d04d838a2207753916c3b4
I20260927 18:41:41.284008 26523 log.cc:1196] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/17493cc2e5d04d838a2207753916c3b4
I20260927 18:41:41.284314 26523 ts_tablet_manager.cc:1939] T 17493cc2e5d04d838a2207753916c3b4 P 8243ccc2001c4efe8144c247db62a157: Deleting consensus metadata
I20260927 18:41:41.284327 26524 ts_tablet_manager.cc:1939] T 17493cc2e5d04d838a2207753916c3b4 P e4538361d248498bab4699b9d945e7cc: Deleting consensus metadata
I20260927 18:41:41.284765 26008 catalog_manager.cc:4841] TS 8243ccc2001c4efe8144c247db62a157 (127.11.251.194:37107): tablet 17493cc2e5d04d838a2207753916c3b4 (table test-table [id=f4c63761b1d4454fa11f1d0a11bc52a0]) successfully deleted
I20260927 18:41:41.284878 26008 catalog_manager.cc:4841] TS e4538361d248498bab4699b9d945e7cc (127.11.251.195:35517): tablet 17493cc2e5d04d838a2207753916c3b4 (table test-table [id=f4c63761b1d4454fa11f1d0a11bc52a0]) successfully deleted
W20260927 18:41:41.288807 26010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.251.193:43337 (error 108)
I20260927 18:41:41.289011 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 26220
I20260927 18:41:41.289968 26495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.289804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:43337 (local address 127.0.0.1:37712)
0927 18:41:41.289870 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:41.289920 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.193:43337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260927 18:41:41.290330 26010 catalog_manager.cc:4584] TS 057edf154c3a45a2bfc592ef9e5bce20 (127.11.251.193:43337): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 17493cc2e5d04d838a2207753916c3b4: Network error: Client connection negotiation failed: client connection to 127.11.251.193:43337: connect: Connection refused (error 111)
I20260927 18:41:41.294936 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 26354
I20260927 18:41:41.300256 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 25992
I20260927 18:41:41.306505 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:42519
--webserver_interface=127.11.251.254
--webserver_port=43115
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:42519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:41.397974 26526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:41.398393 26526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:41.398432 26526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:41.399982 26526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:41.400027 26526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:41.400041 26526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:41.400053 26526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:41.400066 26526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:41.401557 26526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:42519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:42519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=43115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:41.401863 26526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:41.402082 26526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:41.411724 26533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:41.411801 26532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:41.411906 26526 server_base.cc:1034] running on GCE node
W20260927 18:41:41.412012 26535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:41.412318 26526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:41.412667 26526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:41.413828 26526 hybrid_clock.cc:648] HybridClock initialized: now 1790534501413807 us; error 38 us; skew 500 ppm
Sep 27 18:41:41 dist-test-slave-z47j krb5kdc[25979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534501, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:41.416596 26526 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:41:41.417081 26526 webserver.cc:460] Webserver started at http://127.11.251.254:43115/ using document root <none> and password file <none>
I20260927 18:41:41.417341 26526 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:41.417434 26526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:41.419015 26526 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:41:41.419796 26542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:41.419994 26526 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:41:41.420109 26526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "69a1fe23a8734b86b7b2b98263fe2e7f"
format_stamp: "Formatted at 2026-09-27 18:41:40 on dist-test-slave-z47j"
server_key: "13bc6a178f0709e8ac887d0cca74fc96"
server_key_iv: "df21fe845dabfaaa9cf3f6dfe62a9137"
server_key_version: "encryptionkey@0"
I20260927 18:41:41.420567 26526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:41.442752 26526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:41.445371 26526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:41.445549 26526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:41.450330 26526 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:42519
I20260927 18:41:41.450366 26594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:42519 every 8 connection(s)
I20260927 18:41:41.450827 26526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:41.452561 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 26526
I20260927 18:41:41.452806 26595 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:41:41.452943 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:43337
--local_ip_for_outbound_sockets=127.11.251.193
--tserver_master_addrs=127.11.251.254:42519
--webserver_port=35185
--webserver_interface=127.11.251.193
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:41:41.454000 26595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f: Bootstrap starting.
I20260927 18:41:41.464458 26595 log.cc:824] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:41.466763 26595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:41:41.467132 26595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f: Bootstrap complete.
I20260927 18:41:41.469812 26595 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 42519 } }
I20260927 18:41:41.470088 26595 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69a1fe23a8734b86b7b2b98263fe2e7f, State: Initialized, Role: FOLLOWER
I20260927 18:41:41.470306 26595 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 42519 } }
I20260927 18:41:41.470405 26595 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:41.470467 26595 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:41.470517 26595 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 1 FOLLOWER]: Advancing to term 2
I20260927 18:41:41.471674 26595 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 42519 } }
I20260927 18:41:41.471805 26595 leader_election.cc:302] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69a1fe23a8734b86b7b2b98263fe2e7f; no voters:
I20260927 18:41:41.472029 26595 leader_election.cc:288] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 18:41:41.472069 26599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 2 FOLLOWER]: Leader election won for term 2
I20260927 18:41:41.472193 26599 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 2 LEADER]: Becoming Leader. State: Replica: 69a1fe23a8734b86b7b2b98263fe2e7f, State: Running, Role: LEADER
I20260927 18:41:41.472296 26599 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 42519 } }
I20260927 18:41:41.472348 26595 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:41.472582 26599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69a1fe23a8734b86b7b2b98263fe2e7f. Latest consensus state: current_term: 2 leader_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 42519 } } }
I20260927 18:41:41.472645 26599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:41.472772 26600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 42519 } } }
I20260927 18:41:41.472831 26600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:41.473242 26603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:41.474150 26603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:41.474239 26603 catalog_manager.cc:1259] Loaded cluster ID: 2f8b7cedff814801aee6d23b2b5583e8
I20260927 18:41:41.474262 26603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:41.474529 26603 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260927 18:41:41.478978 26603 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260927 18:41:41.481496 26603 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:41.481725 26603 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f: Loaded TSK: 0
I20260927 18:41:41.481941 26603 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 18:41:41.547724 26597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:41.547974 26597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:41.548000 26597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:41.549237 26597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:41.549280 26597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:41.549310 26597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:41:41.550854 26597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:43337
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=35185
--tserver_master_addrs=127.11.251.254:42519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:41.551126 26597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:41.551406 26597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:41.557082 26625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:41.557197 26623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:41.557101 26622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:41.557256 26597 server_base.cc:1034] running on GCE node
I20260927 18:41:41.557505 26597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:41.557801 26597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:41.558975 26597 hybrid_clock.cc:648] HybridClock initialized: now 1790534501558952 us; error 45 us; skew 500 ppm
Sep 27 18:41:41 dist-test-slave-z47j krb5kdc[25979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534501, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:41.562019 26597 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:41:41.562598 26597 webserver.cc:460] Webserver started at http://127.11.251.193:35185/ using document root <none> and password file <none>
I20260927 18:41:41.562867 26597 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:41.562948 26597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:41.564811 26597 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:41:41.565693 26632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:41.565936 26597 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 18:41:41.566032 26597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "057edf154c3a45a2bfc592ef9e5bce20"
format_stamp: "Formatted at 2026-09-27 18:41:40 on dist-test-slave-z47j"
server_key: "3709558c78db446ea4dacf808da7c95e"
server_key_iv: "5d85c5cbbe6d07dcffa5bca48e0968e5"
server_key_version: "encryptionkey@0"
I20260927 18:41:41.566570 26597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:41.582715 26597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:41.585544 26597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:41.585754 26597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:41.586079 26597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:41.586634 26639 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 18:41:41.588234 26597 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 18:41:41.588296 26597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260927 18:41:41.588336 26597 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 18:41:41.589298 26597 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 18:41:41.589342 26597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260927 18:41:41.589445 26639 tablet_bootstrap.cc:492] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20: Bootstrap starting.
I20260927 18:41:41.598143 26597 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:43337
I20260927 18:41:41.598481 26746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:43337 every 8 connection(s)
I20260927 18:41:41.598691 26597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:41:41.599421 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 26597
I20260927 18:41:41.599748 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:37107
--local_ip_for_outbound_sockets=127.11.251.194
--tserver_master_addrs=127.11.251.254:42519
--webserver_port=40725
--webserver_interface=127.11.251.194
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:41:41.604261 26639 log.cc:824] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:41.607458 26639 tablet_bootstrap.cc:492] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:41:41.607852 26639 tablet_bootstrap.cc:492] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20: Bootstrap complete.
I20260927 18:41:41.609170 26639 ts_tablet_manager.cc:1397] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.008s
I20260927 18:41:41.611123 26639 raft_consensus.cc:348] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } }
I20260927 18:41:41.611515 26639 raft_consensus.cc:729] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 057edf154c3a45a2bfc592ef9e5bce20, State: Initialized, Role: FOLLOWER
I20260927 18:41:41.611670 26639 consensus_queue.cc:260] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } }
I20260927 18:41:41.611974 26639 ts_tablet_manager.cc:1428] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
Sep 27 18:41:41 dist-test-slave-z47j krb5kdc[25979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534501, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:41:41.617158 26750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.599968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:60181 (local address 127.11.251.254:42519)
0927 18:41:41.604570 (+ 4602us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:41.604585 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:41.604649 (+ 64us) server_negotiation.cc:407] Connection header received
0927 18:41:41.604829 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:41.604838 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:41.604912 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:41.606133 (+ 1221us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:41.606252 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.607866 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.609637 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.610244 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.612172 (+ 1928us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.613436 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:41.613441 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:41.613441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:41.613460 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:41.615352 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:41.615803 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:41.615807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:41.615807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:41.615911 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:41.616179 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:41.616180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:41.616180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:41.616422 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:41.616514 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:41.616813 (+ 299us) server_negotiation.cc:299] Negotiation successful
0927 18:41:41.616947 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4523,"thread_start_us":91,"threads_started":1}
I20260927 18:41:41.617157 26749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.599571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.193:60181)
0927 18:41:41.600051 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:41.600107 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:41.600968 (+ 861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:41.605137 (+ 4169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:41.605143 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:41.605675 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:41.605975 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.605984 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.608042 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.608043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.609495 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.609497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.610367 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.610368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.610522 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.611254 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:41.611266 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:41.613294 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:41.615459 (+ 2165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:41.615466 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:41.615469 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:41.615702 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:41.615988 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:41.615988 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:41.615989 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:41.616100 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:41.616504 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:41.616510 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:41.616649 (+ 139us) client_negotiation.cc:769] Sending connection context
0927 18:41:41.616761 (+ 112us) client_negotiation.cc:240] Negotiation successful
0927 18:41:41.616947 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":108,"threads_started":1}
I20260927 18:41:41.618384 26747 heartbeater.cc:347] Connected to a master server at 127.11.251.254:42519
I20260927 18:41:41.618510 26747 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:41.618723 26747 heartbeater.cc:511] Master 127.11.251.254:42519 requested a full tablet report, sending...
I20260927 18:41:41.619536 26559 ts_manager.cc:194] Registered new tserver with Master: 057edf154c3a45a2bfc592ef9e5bce20 (127.11.251.193:43337)
I20260927 18:41:41.620674 26559 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:60181
I20260927 18:41:41.621565 26747 heartbeater.cc:503] Master 127.11.251.254:42519 was elected leader, sending a full tablet report...
I20260927 18:41:41.629129 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.620194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:43337 (local address 127.0.0.1:37724)
0927 18:41:41.625754 (+ 5560us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:41.625822 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:41.625902 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:41.626194 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:41.626197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:41.626200 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:41.626293 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.626295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.627240 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.627241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.628078 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.628080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.628773 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.628774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.628867 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.628884 (+ 17us) client_negotiation.cc:769] Sending connection context
0927 18:41:41.628977 (+ 93us) client_negotiation.cc:240] Negotiation successful
0927 18:41:41.629044 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5524,"thread_start_us":95,"threads_started":1}
I20260927 18:41:41.629129 26755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.621007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37724 (local address 127.11.251.193:43337)
0927 18:41:41.626037 (+ 5030us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:41.626040 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:41.626048 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:41:41.626076 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:41.626077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:41.626091 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:41.626188 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:41.626380 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.627155 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.628185 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.628670 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.628774 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.628840 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:41.628997 (+ 157us) server_negotiation.cc:299] Negotiation successful
0927 18:41:41.629045 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4998,"thread_start_us":98,"threads_started":1}
I20260927 18:41:41.630482 26681 tablet_service.cc:1514] Processing DeleteTablet for tablet 17493cc2e5d04d838a2207753916c3b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:41 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:37724
I20260927 18:41:41.631101 26756 tablet_replica.cc:330] stopping tablet replica
I20260927 18:41:41.631232 26756 raft_consensus.cc:2229] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:41:41.631340 26756 raft_consensus.cc:2258] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:41:41.631700 26756 ts_tablet_manager.cc:1905] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:41:41.633805 26756 ts_tablet_manager.cc:1918] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:41:41.633894 26756 log.cc:1196] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/wal/wals/17493cc2e5d04d838a2207753916c3b4
I20260927 18:41:41.634133 26756 ts_tablet_manager.cc:1939] T 17493cc2e5d04d838a2207753916c3b4 P 057edf154c3a45a2bfc592ef9e5bce20: Deleting consensus metadata
I20260927 18:41:41.634687 26545 catalog_manager.cc:4841] TS 057edf154c3a45a2bfc592ef9e5bce20 (127.11.251.193:43337): tablet 17493cc2e5d04d838a2207753916c3b4 (table test-table [id=f4c63761b1d4454fa11f1d0a11bc52a0]) successfully deleted
W20260927 18:41:41.699715 26751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:41.699955 26751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:41.699985 26751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:41.701231 26751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:41.701275 26751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:41.701309 26751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:41:41.702898 26751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:37107
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=40725
--tserver_master_addrs=127.11.251.254:42519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:41.703191 26751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:41.703472 26751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:41.709136 26762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:41.709137 26761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:41.709144 26764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:41.709461 26751 server_base.cc:1034] running on GCE node
I20260927 18:41:41.709739 26751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:41.709997 26751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:41.711156 26751 hybrid_clock.cc:648] HybridClock initialized: now 1790534501711124 us; error 52 us; skew 500 ppm
Sep 27 18:41:41 dist-test-slave-z47j krb5kdc[25979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534501, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:41.714046 26751 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:41:41.714579 26751 webserver.cc:460] Webserver started at http://127.11.251.194:40725/ using document root <none> and password file <none>
I20260927 18:41:41.714848 26751 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:41.714936 26751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:41.716405 26751 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:41:41.717095 26771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:41.717327 26751 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:41:41.717427 26751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "8243ccc2001c4efe8144c247db62a157"
format_stamp: "Formatted at 2026-09-27 18:41:40 on dist-test-slave-z47j"
server_key: "9453a5bf33e1c67688412a08c751e9e6"
server_key_iv: "e94efb69fa4c6ae49f14e2b9a8281543"
server_key_version: "encryptionkey@0"
I20260927 18:41:41.717924 26751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:41.727536 26751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:41.729547 26751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:41.729751 26751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:41.729998 26751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:41.730329 26751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:41.730383 26751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:41.730430 26751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:41.730455 26751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:41.737627 26751 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:37107
I20260927 18:41:41.737679 26884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:37107 every 8 connection(s)
I20260927 18:41:41.738099 26751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
Sep 27 18:41:41 dist-test-slave-z47j krb5kdc[25979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534501, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:41:41.746843 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 26751
I20260927 18:41:41.747159 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:35517
--local_ip_for_outbound_sockets=127.11.251.195
--tserver_master_addrs=127.11.251.254:42519
--webserver_port=45865
--webserver_interface=127.11.251.195
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:41:41.749331 26750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.738967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:49397 (local address 127.11.251.254:42519)
0927 18:41:41.739042 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:41.739044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:41.739277 (+ 233us) server_negotiation.cc:407] Connection header received
0927 18:41:41.739910 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:41.739912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:41.739930 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:41.740011 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:41.741063 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.741716 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.742792 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.743037 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.743129 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.745790 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:41.745792 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:41.745793 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:41.745802 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:41.747484 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:41.748068 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:41.748069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:41.748069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:41.748127 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:41.748416 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:41.748416 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:41.748417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:41.748539 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:41.748595 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:41.749081 (+ 486us) server_negotiation.cc:299] Negotiation successful
0927 18:41:41.749202 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:41:41.749351 26887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.738950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.194:49397)
0927 18:41:41.739145 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:41.739186 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:41.739792 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:41.740090 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:41.740094 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:41.740722 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:41.740952 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.740957 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.741814 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.741815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.742693 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.742695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.743139 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.743141 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.743279 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.743828 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:41.743836 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:41.745684 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:41.747568 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:41.747575 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:41.747579 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:41.747963 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:41.748196 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:41.748197 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:41.748197 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:41.748347 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:41.748605 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:41.748611 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:41.748800 (+ 189us) client_negotiation.cc:769] Sending connection context
0927 18:41:41.748946 (+ 146us) client_negotiation.cc:240] Negotiation successful
0927 18:41:41.749152 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":104,"threads_started":1}
I20260927 18:41:41.749997 26885 heartbeater.cc:347] Connected to a master server at 127.11.251.254:42519
I20260927 18:41:41.750120 26885 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:41.750335 26885 heartbeater.cc:511] Master 127.11.251.254:42519 requested a full tablet report, sending...
I20260927 18:41:41.750828 26559 ts_manager.cc:194] Registered new tserver with Master: 8243ccc2001c4efe8144c247db62a157 (127.11.251.194:37107)
I20260927 18:41:41.751436 26559 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:49397
W20260927 18:41:41.840801 26888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:41.841028 26888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:41.841053 26888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:41.842289 26888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:41.842332 26888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:41.842360 26888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:41:41.843842 26888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:35517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=45865
--tserver_master_addrs=127.11.251.254:42519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:41.844048 26888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:41.844242 26888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:41.849893 26893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:41.849898 26894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:41.849898 26896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:41.850209 26888 server_base.cc:1034] running on GCE node
I20260927 18:41:41.850461 26888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:41.850729 26888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:41.851910 26888 hybrid_clock.cc:648] HybridClock initialized: now 1790534501851889 us; error 55 us; skew 500 ppm
Sep 27 18:41:41 dist-test-slave-z47j krb5kdc[25979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534501, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:41.854830 26888 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:41:41.855337 26888 webserver.cc:460] Webserver started at http://127.11.251.195:45865/ using document root <none> and password file <none>
I20260927 18:41:41.855545 26888 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:41.855609 26888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:41.857050 26888 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:41:41.857692 26903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:41.857877 26888 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 18:41:41.857975 26888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "e4538361d248498bab4699b9d945e7cc"
format_stamp: "Formatted at 2026-09-27 18:41:41 on dist-test-slave-z47j"
server_key: "f890584ae50388a89d665b9993a4c687"
server_key_iv: "e323d53d0b4869a4ca05c8ae4d82d737"
server_key_version: "encryptionkey@0"
I20260927 18:41:41.858363 26888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:41.898154 26888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:41.900425 26888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:41.900553 26888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:41.900802 26888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:41.901209 26888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:41.901247 26888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:41.901273 26888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:41.901285 26888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:41.908145 26888 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:35517
I20260927 18:41:41.908262 27016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:35517 every 8 connection(s)
I20260927 18:41:41.908629 26888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:41:41.915010 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 26888
Sep 27 18:41:41 dist-test-slave-z47j krb5kdc[25979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534501, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:41:41.919457 26750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.909501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:53547 (local address 127.11.251.254:42519)
0927 18:41:41.909593 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:41.909596 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:41.909868 (+ 272us) server_negotiation.cc:407] Connection header received
0927 18:41:41.910438 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:41.910439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:41.910452 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:41.910514 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:41.911549 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.912076 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.913118 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.913314 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.913392 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.916091 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:41.916092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:41.916092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:41.916098 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:41.917857 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:41.918385 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:41.918385 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:41.918386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:41.918437 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:41.918719 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:41.918720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:41.918720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:41.918824 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:41.918875 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:41.919244 (+ 369us) server_negotiation.cc:299] Negotiation successful
0927 18:41:41.919327 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:41:41.919653 27019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.909472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.195:53547)
0927 18:41:41.909705 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:41.909753 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:41.910341 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:41.910575 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:41.910578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:41.911233 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:41.911467 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.911472 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.912157 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.912158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.913032 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.913034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.913397 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.913398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.913522 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.914055 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:41.914064 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:41.916007 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:41.917962 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:41.917969 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:41.917972 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:41.918298 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:41.918507 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:41.918508 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:41.918508 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:41.918652 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:41.918893 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:41.918899 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:41.919110 (+ 211us) client_negotiation.cc:769] Sending connection context
0927 18:41:41.919205 (+ 95us) client_negotiation.cc:240] Negotiation successful
0927 18:41:41.919372 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":128,"threads_started":1}
I20260927 18:41:41.920445 27017 heartbeater.cc:347] Connected to a master server at 127.11.251.254:42519
I20260927 18:41:41.920557 27017 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:41.920782 27017 heartbeater.cc:511] Master 127.11.251.254:42519 requested a full tablet report, sending...
I20260927 18:41:41.921548 26559 ts_manager.cc:194] Registered new tserver with Master: e4538361d248498bab4699b9d945e7cc (127.11.251.195:35517)
I20260927 18:41:41.922149 26559 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:53547
I20260927 18:41:41.923563 27021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.915487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48766 (local address 127.11.251.254:42519)
0927 18:41:41.915639 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:41.915641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:41.915651 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:41:41.916172 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:41.916173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:41.916186 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:41.916272 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:41.917054 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.917881 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.919063 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.919312 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.919380 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.920915 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:41.920916 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:41.920917 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:41.920926 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:41.922373 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:41.922700 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:41.922701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:41.922702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:41.922764 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:41.922991 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:41.923009 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:41.923009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:41.923129 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:41.923331 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:41.923368 (+ 37us) server_negotiation.cc:299] Negotiation successful
0927 18:41:41.923449 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260927 18:41:41.924394 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:41:41.927083 27021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.924766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48770 (local address 127.11.251.254:42519)
0927 18:41:41.924821 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:41.924822 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:41.924963 (+ 141us) server_negotiation.cc:407] Connection header received
0927 18:41:41.925023 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:41.925023 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:41.925056 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:41.925114 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:41:41.925202 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.925723 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.926418 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.926602 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.926650 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.926800 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:41:41.926895 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:41:41.926953 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:41.926983 (+ 30us) server_negotiation.cc:299] Negotiation successful
0927 18:41:41.926999 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:41:41.927410 26559 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:48770:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:41:41.929312 26559 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:41:41.931757 26681 tablet_service.cc:1467] Processing CreateTablet for tablet f2441f107302416085f4eca7a0e8c11a (DEFAULT_TABLE table=test-table [id=afb8cfb3bdc644c983de4b923bf9b871]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:41.932013 26681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2441f107302416085f4eca7a0e8c11a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:41.933954 26639 tablet_bootstrap.cc:492] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20: Bootstrap starting.
I20260927 18:41:41.934309 26639 tablet_bootstrap.cc:654] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:41.934985 26639 tablet_bootstrap.cc:492] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20: No bootstrap required, opened a new log
I20260927 18:41:41.935052 26639 ts_tablet_manager.cc:1397] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260927 18:41:41.935189 26639 raft_consensus.cc:348] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } }
I20260927 18:41:41.935292 26639 raft_consensus.cc:374] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:41.935345 26639 raft_consensus.cc:729] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 057edf154c3a45a2bfc592ef9e5bce20, State: Initialized, Role: FOLLOWER
I20260927 18:41:41.935407 26639 consensus_queue.cc:260] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } }
I20260927 18:41:41.935513 26639 ts_tablet_manager.cc:1428] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:41.935563 27023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.931479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:35517 (local address 127.0.0.1:40316)
0927 18:41:41.931656 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:41.931666 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:41.931723 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:41.932172 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:41.932173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:41.932176 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:41.932220 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.932222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.933126 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.933127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.934290 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.934293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.935267 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.935268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.935337 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.935341 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:41:41.935417 (+ 76us) client_negotiation.cc:240] Negotiation successful
0927 18:41:41.935465 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":97,"threads_started":1}
I20260927 18:41:41.935791 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.931356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:37107 (local address 127.0.0.1:35712)
0927 18:41:41.931423 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:41.931435 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:41.931472 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:41.932179 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:41.932180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:41.932182 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:41.932227 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.932228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.933572 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.933588 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.934616 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:41.934620 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.935514 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.935515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:41.935588 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.935591 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:41:41.935622 (+ 31us) client_negotiation.cc:240] Negotiation successful
0927 18:41:41.935672 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 18:41:41.935882 27025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.931726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40316 (local address 127.11.251.195:35517)
0927 18:41:41.931938 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:41.931943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:41.931953 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:41:41.931993 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:41.931995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:41.932012 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:41.932130 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:41.932303 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.933019 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.934393 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.935174 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.935268 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.935328 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:41.935456 (+ 128us) server_negotiation.cc:299] Negotiation successful
0927 18:41:41.935782 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":104,"threads_started":1}
I20260927 18:41:41.936206 27024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:41.931511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35712 (local address 127.11.251.194:37107)
0927 18:41:41.931962 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:41.931967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:41.931978 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:41:41.932025 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:41.932027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:41.932043 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:41.932357 (+ 314us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:41.932449 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.933454 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.934703 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:41.935416 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:41.935965 (+ 549us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:41.936039 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:41.936064 (+ 25us) server_negotiation.cc:299] Negotiation successful
0927 18:41:41.936110 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":168,"threads_started":1}
I20260927 18:41:41.936611 26951 tablet_service.cc:1467] Processing CreateTablet for tablet f2441f107302416085f4eca7a0e8c11a (DEFAULT_TABLE table=test-table [id=afb8cfb3bdc644c983de4b923bf9b871]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:41.936887 26819 tablet_service.cc:1467] Processing CreateTablet for tablet f2441f107302416085f4eca7a0e8c11a (DEFAULT_TABLE table=test-table [id=afb8cfb3bdc644c983de4b923bf9b871]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:41.937009 26951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2441f107302416085f4eca7a0e8c11a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:41.937206 26819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2441f107302416085f4eca7a0e8c11a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:41.941009 27027 tablet_bootstrap.cc:492] T f2441f107302416085f4eca7a0e8c11a P e4538361d248498bab4699b9d945e7cc: Bootstrap starting.
I20260927 18:41:41.941766 27028 tablet_bootstrap.cc:492] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157: Bootstrap starting.
I20260927 18:41:41.941813 27027 tablet_bootstrap.cc:654] T f2441f107302416085f4eca7a0e8c11a P e4538361d248498bab4699b9d945e7cc: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:41.942484 27027 log.cc:824] T f2441f107302416085f4eca7a0e8c11a P e4538361d248498bab4699b9d945e7cc: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:41.942544 27028 tablet_bootstrap.cc:654] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:41.943138 27028 log.cc:824] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:41.943400 27027 tablet_bootstrap.cc:492] T f2441f107302416085f4eca7a0e8c11a P e4538361d248498bab4699b9d945e7cc: No bootstrap required, opened a new log
I20260927 18:41:41.943480 27027 ts_tablet_manager.cc:1397] T f2441f107302416085f4eca7a0e8c11a P e4538361d248498bab4699b9d945e7cc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:41:41.943964 27028 tablet_bootstrap.cc:492] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157: No bootstrap required, opened a new log
I20260927 18:41:41.944041 27028 ts_tablet_manager.cc:1397] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:41:41.945773 27027 raft_consensus.cc:348] T f2441f107302416085f4eca7a0e8c11a P e4538361d248498bab4699b9d945e7cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } }
I20260927 18:41:41.945971 27027 raft_consensus.cc:374] T f2441f107302416085f4eca7a0e8c11a P e4538361d248498bab4699b9d945e7cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:41.946028 27027 raft_consensus.cc:729] T f2441f107302416085f4eca7a0e8c11a P e4538361d248498bab4699b9d945e7cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4538361d248498bab4699b9d945e7cc, State: Initialized, Role: FOLLOWER
I20260927 18:41:41.946174 27027 consensus_queue.cc:260] T f2441f107302416085f4eca7a0e8c11a P e4538361d248498bab4699b9d945e7cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } }
I20260927 18:41:41.946295 27028 raft_consensus.cc:348] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } }
I20260927 18:41:41.946465 27028 raft_consensus.cc:374] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:41.946518 27028 raft_consensus.cc:729] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8243ccc2001c4efe8144c247db62a157, State: Initialized, Role: FOLLOWER
I20260927 18:41:41.946530 27027 ts_tablet_manager.cc:1428] T f2441f107302416085f4eca7a0e8c11a P e4538361d248498bab4699b9d945e7cc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:41:41.946600 27017 heartbeater.cc:503] Master 127.11.251.254:42519 was elected leader, sending a full tablet report...
I20260927 18:41:41.946640 27028 consensus_queue.cc:260] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } }
I20260927 18:41:41.946949 27028 ts_tablet_manager.cc:1428] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:41:41.946990 26885 heartbeater.cc:503] Master 127.11.251.254:42519 was elected leader, sending a full tablet report...
W20260927 18:41:41.988808 26886 tablet.cc:2369] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:41:42.050025 26753 raft_consensus.cc:482] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:41:42.050275 26753 raft_consensus.cc:504] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } }
I20260927 18:41:42.050738 26753 leader_election.cc:288] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8243ccc2001c4efe8144c247db62a157 (127.11.251.194:37107), e4538361d248498bab4699b9d945e7cc (127.11.251.195:35517)
I20260927 18:41:42.054559 27024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:42.051071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:54957 (local address 127.11.251.194:37107)
0927 18:41:42.051123 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:42.051125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:42.051137 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:41:42.051176 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:42.051177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:42.051189 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:42.051269 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:42.051467 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:42.052152 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:42.053569 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:42.054153 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:42.054234 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:42.054245 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:42.054420 (+ 175us) server_negotiation.cc:299] Negotiation successful
0927 18:41:42.054464 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:41:42.054574 26749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:42.050970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:37107 (local address 127.11.251.193:54957)
0927 18:41:42.051037 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:42.051052 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:42.051105 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:42.051285 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:42.051286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:42.051289 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:42.051366 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:42.051369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:42.052237 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:42.052238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:42.053445 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:42.053447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:42.054259 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:42.054260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:42.054332 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:42.054336 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:41:42.054398 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 18:41:42.054442 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 18:41:42.054977 27033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:42.051107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:35517 (local address 127.11.251.193:57685)
0927 18:41:42.051332 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:42.051362 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:42.051438 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:42.051573 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:42.051573 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:42.051575 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:42.051918 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:42.051919 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:42.052656 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:42.052657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:42.053777 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:42.053779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:42.054601 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:42.054602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:42.054693 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:42.054697 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:41:42.054805 (+ 108us) client_negotiation.cc:240] Negotiation successful
0927 18:41:42.054863 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":156,"threads_started":1}
I20260927 18:41:42.054991 27025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:42.051258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:57685 (local address 127.11.251.195:35517)
0927 18:41:42.051321 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:42.051323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:42.051448 (+ 125us) server_negotiation.cc:407] Connection header received
0927 18:41:42.051488 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:42.051490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:42.051500 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:42.051562 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:42.052005 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:42.052565 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:42.053871 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:42.054499 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:42.054574 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:42.054583 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:42.054856 (+ 273us) server_negotiation.cc:299] Negotiation successful
0927 18:41:42.054910 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:41:42.055025 26839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2441f107302416085f4eca7a0e8c11a" candidate_uuid: "057edf154c3a45a2bfc592ef9e5bce20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8243ccc2001c4efe8144c247db62a157" is_pre_election: true
I20260927 18:41:42.055236 26839 raft_consensus.cc:2454] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 057edf154c3a45a2bfc592ef9e5bce20 in term 0.
I20260927 18:41:42.055431 26971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2441f107302416085f4eca7a0e8c11a" candidate_uuid: "057edf154c3a45a2bfc592ef9e5bce20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4538361d248498bab4699b9d945e7cc" is_pre_election: true
I20260927 18:41:42.055492 26633 leader_election.cc:302] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 057edf154c3a45a2bfc592ef9e5bce20, 8243ccc2001c4efe8144c247db62a157; no voters:
I20260927 18:41:42.055632 26971 raft_consensus.cc:2454] T f2441f107302416085f4eca7a0e8c11a P e4538361d248498bab4699b9d945e7cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 057edf154c3a45a2bfc592ef9e5bce20 in term 0.
I20260927 18:41:42.055646 26753 raft_consensus.cc:2781] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:41:42.055727 26753 raft_consensus.cc:482] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:41:42.055764 26753 raft_consensus.cc:3037] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:42.056635 26753 raft_consensus.cc:504] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } }
I20260927 18:41:42.056787 26753 leader_election.cc:288] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 1 election: Requested vote from peers 8243ccc2001c4efe8144c247db62a157 (127.11.251.194:37107), e4538361d248498bab4699b9d945e7cc (127.11.251.195:35517)
I20260927 18:41:42.056985 26839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2441f107302416085f4eca7a0e8c11a" candidate_uuid: "057edf154c3a45a2bfc592ef9e5bce20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8243ccc2001c4efe8144c247db62a157"
I20260927 18:41:42.057039 26971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2441f107302416085f4eca7a0e8c11a" candidate_uuid: "057edf154c3a45a2bfc592ef9e5bce20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4538361d248498bab4699b9d945e7cc"
I20260927 18:41:42.057070 26839 raft_consensus.cc:3037] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:42.057132 26971 raft_consensus.cc:3037] T f2441f107302416085f4eca7a0e8c11a P e4538361d248498bab4699b9d945e7cc [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:42.058004 26971 raft_consensus.cc:2454] T f2441f107302416085f4eca7a0e8c11a P e4538361d248498bab4699b9d945e7cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 057edf154c3a45a2bfc592ef9e5bce20 in term 1.
I20260927 18:41:42.058178 26839 raft_consensus.cc:2454] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 057edf154c3a45a2bfc592ef9e5bce20 in term 1.
I20260927 18:41:42.058290 26633 leader_election.cc:302] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 057edf154c3a45a2bfc592ef9e5bce20, e4538361d248498bab4699b9d945e7cc; no voters:
I20260927 18:41:42.058403 26753 raft_consensus.cc:2781] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:42.058629 26753 raft_consensus.cc:686] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 LEADER]: Becoming Leader. State: Replica: 057edf154c3a45a2bfc592ef9e5bce20, State: Running, Role: LEADER
I20260927 18:41:42.058789 26753 consensus_queue.cc:237] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } }
I20260927 18:41:42.059798 26559 catalog_manager.cc:5495] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 reported cstate change: term changed from 0 to 1, leader changed from <none> to 057edf154c3a45a2bfc592ef9e5bce20 (127.11.251.193). New cstate: current_term: 1 leader_uuid: "057edf154c3a45a2bfc592ef9e5bce20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } health_report { overall_health: HEALTHY } } }
I20260927 18:41:42.082680 26755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:42.080084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37732 (local address 127.11.251.193:43337)
0927 18:41:42.080151 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:42.080153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:42.080162 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:41:42.080240 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:42.080241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:42.080260 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:42.080329 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:41:42.080448 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:42.081023 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:42.081853 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:42.082074 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:42.082137 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:42.082268 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:41:42.082385 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:41:42.082443 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:42.082539 (+ 96us) server_negotiation.cc:299] Negotiation successful
0927 18:41:42.082558 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:41:42.084606 26971 raft_consensus.cc:1261] T f2441f107302416085f4eca7a0e8c11a P e4538361d248498bab4699b9d945e7cc [term 1 FOLLOWER]: Refusing update from remote peer 057edf154c3a45a2bfc592ef9e5bce20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:42.084632 26839 raft_consensus.cc:1261] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157 [term 1 FOLLOWER]: Refusing update from remote peer 057edf154c3a45a2bfc592ef9e5bce20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:42.085036 27034 consensus_queue.cc:1035] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:42.085246 26753 consensus_queue.cc:1035] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:42.088476 27037 mvcc.cc:205] Tried to move back new op lower bound from 7334029320535822336 to 7334029320433471488. Current Snapshot: MvccSnapshot[applied={T|T < 7334029320535822336 or (T in {7334029320535822336})}]
I20260927 18:41:42.088891 27038 mvcc.cc:205] Tried to move back new op lower bound from 7334029320535822336 to 7334029320433471488. Current Snapshot: MvccSnapshot[applied={T|T < 7334029320535822336}]
I20260927 18:41:42.090489 26559 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:48770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:41:42.090608 26559 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:48770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:41:42.091514 26559 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f: Sending DeleteTablet for 3 replicas of tablet f2441f107302416085f4eca7a0e8c11a
I20260927 18:41:42.091883 26819 tablet_service.cc:1514] Processing DeleteTablet for tablet f2441f107302416085f4eca7a0e8c11a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:42 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:35712
I20260927 18:41:42.091948 26681 tablet_service.cc:1514] Processing DeleteTablet for tablet f2441f107302416085f4eca7a0e8c11a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:42 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:37724
I20260927 18:41:42.091926 26951 tablet_service.cc:1514] Processing DeleteTablet for tablet f2441f107302416085f4eca7a0e8c11a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:42 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:40316
I20260927 18:41:42.092041 26756 tablet_replica.cc:330] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20: stopping tablet replica
I20260927 18:41:42.092109 26756 raft_consensus.cc:2229] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:41:42.092118 27046 tablet_replica.cc:330] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157: stopping tablet replica
I20260927 18:41:42.092219 27046 raft_consensus.cc:2229] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:41:42.092199 26756 raft_consensus.cc:2258] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:41:42.092331 27046 raft_consensus.cc:2258] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:41:42.092595 26756 ts_tablet_manager.cc:1905] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:41:42.092634 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 26597
I20260927 18:41:42.093336 27046 ts_tablet_manager.cc:1905] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:41:42.094404 27047 tablet_replica.cc:330] T f2441f107302416085f4eca7a0e8c11a P e4538361d248498bab4699b9d945e7cc: stopping tablet replica
I20260927 18:41:42.094527 27047 raft_consensus.cc:2229] T f2441f107302416085f4eca7a0e8c11a P e4538361d248498bab4699b9d945e7cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:41:42.094640 27047 raft_consensus.cc:2258] T f2441f107302416085f4eca7a0e8c11a P e4538361d248498bab4699b9d945e7cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:41:42.095223 27047 ts_tablet_manager.cc:1905] T f2441f107302416085f4eca7a0e8c11a P e4538361d248498bab4699b9d945e7cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:41:42.095705 27046 ts_tablet_manager.cc:1918] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:41:42.095784 27046 log.cc:1196] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/f2441f107302416085f4eca7a0e8c11a
I20260927 18:41:42.096076 27046 ts_tablet_manager.cc:1939] T f2441f107302416085f4eca7a0e8c11a P 8243ccc2001c4efe8144c247db62a157: Deleting consensus metadata
I20260927 18:41:42.096524 26543 catalog_manager.cc:4841] TS 8243ccc2001c4efe8144c247db62a157 (127.11.251.194:37107): tablet f2441f107302416085f4eca7a0e8c11a (table test-table [id=afb8cfb3bdc644c983de4b923bf9b871]) successfully deleted
I20260927 18:41:42.096984 27047 ts_tablet_manager.cc:1918] T f2441f107302416085f4eca7a0e8c11a P e4538361d248498bab4699b9d945e7cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:41:42.097049 27047 log.cc:1196] T f2441f107302416085f4eca7a0e8c11a P e4538361d248498bab4699b9d945e7cc: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/f2441f107302416085f4eca7a0e8c11a
I20260927 18:41:42.097345 27047 ts_tablet_manager.cc:1939] T f2441f107302416085f4eca7a0e8c11a P e4538361d248498bab4699b9d945e7cc: Deleting consensus metadata
I20260927 18:41:42.097821 26543 catalog_manager.cc:4841] TS e4538361d248498bab4699b9d945e7cc (127.11.251.195:35517): tablet f2441f107302416085f4eca7a0e8c11a (table test-table [id=afb8cfb3bdc644c983de4b923bf9b871]) successfully deleted
W20260927 18:41:42.099869 26545 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.251.193:43337 (error 108)
I20260927 18:41:42.100430 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:42.100283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:43337 (local address 127.0.0.1:37742)
0927 18:41:42.100352 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:42.100397 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.193:43337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260927 18:41:42.100538 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 26751
W20260927 18:41:42.100557 26545 catalog_manager.cc:4584] TS 057edf154c3a45a2bfc592ef9e5bce20 (127.11.251.193:43337): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f2441f107302416085f4eca7a0e8c11a: Network error: Client connection negotiation failed: client connection to 127.11.251.193:43337: connect: Connection refused (error 111)
I20260927 18:41:42.106607 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 26888
I20260927 18:41:42.111646 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 26526
I20260927 18:41:42.117864 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:42519
--webserver_interface=127.11.251.254
--webserver_port=43115
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:42519
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:42.207854 27049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:42.208096 27049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:42.208130 27049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:42.209312 27049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:42.209376 27049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:42.209401 27049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:42.209432 27049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:42.209453 27049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:42.211041 27049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:42519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:42519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=43115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:42.211336 27049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:42.211607 27049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:42.216748 27054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:42.216722 27057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:42.216924 27049 server_base.cc:1034] running on GCE node
W20260927 18:41:42.216753 27055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:42.217303 27049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:42.217643 27049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:42.218837 27049 hybrid_clock.cc:648] HybridClock initialized: now 1790534502218820 us; error 41 us; skew 500 ppm
Sep 27 18:41:42 dist-test-slave-z47j krb5kdc[25979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534502, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:42.221508 27049 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:41:42.222040 27049 webserver.cc:460] Webserver started at http://127.11.251.254:43115/ using document root <none> and password file <none>
I20260927 18:41:42.222296 27049 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:42.222370 27049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:42.223760 27049 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:41:42.224439 27064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:42.224613 27049 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:42.224716 27049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "69a1fe23a8734b86b7b2b98263fe2e7f"
format_stamp: "Formatted at 2026-09-27 18:41:40 on dist-test-slave-z47j"
server_key: "13bc6a178f0709e8ac887d0cca74fc96"
server_key_iv: "df21fe845dabfaaa9cf3f6dfe62a9137"
server_key_version: "encryptionkey@0"
I20260927 18:41:42.225109 27049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:42.244489 27049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:42.247087 27049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:42.247257 27049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:42.252092 27049 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:42519
I20260927 18:41:42.252131 27116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:42519 every 8 connection(s)
I20260927 18:41:42.252566 27049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:42.253459 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 27049
I20260927 18:41:42.253892 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:43337
--local_ip_for_outbound_sockets=127.11.251.193
--tserver_master_addrs=127.11.251.254:42519
--webserver_port=35185
--webserver_interface=127.11.251.193
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:41:42.254935 27117 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:41:42.256072 27117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f: Bootstrap starting.
I20260927 18:41:42.266510 27117 log.cc:824] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:42.269315 27117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:41:42.269712 27117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f: Bootstrap complete.
I20260927 18:41:42.272266 27117 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 42519 } }
I20260927 18:41:42.272540 27117 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69a1fe23a8734b86b7b2b98263fe2e7f, State: Initialized, Role: FOLLOWER
I20260927 18:41:42.272730 27117 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 42519 } }
I20260927 18:41:42.272794 27117 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:42.272858 27117 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:42.272907 27117 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 2 FOLLOWER]: Advancing to term 3
I20260927 18:41:42.274075 27117 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 42519 } }
I20260927 18:41:42.274168 27117 leader_election.cc:302] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69a1fe23a8734b86b7b2b98263fe2e7f; no voters:
I20260927 18:41:42.274416 27117 leader_election.cc:288] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [CANDIDATE]: Term 3 election: Requested vote from peers
I20260927 18:41:42.274456 27121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 3 FOLLOWER]: Leader election won for term 3
I20260927 18:41:42.274580 27121 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [term 3 LEADER]: Becoming Leader. State: Replica: 69a1fe23a8734b86b7b2b98263fe2e7f, State: Running, Role: LEADER
I20260927 18:41:42.274676 27121 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 42519 } }
I20260927 18:41:42.274713 27117 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:42.274988 27121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69a1fe23a8734b86b7b2b98263fe2e7f. Latest consensus state: current_term: 3 leader_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 42519 } } }
I20260927 18:41:42.275071 27121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:42.274988 27122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a1fe23a8734b86b7b2b98263fe2e7f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 42519 } } }
I20260927 18:41:42.275107 27122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69a1fe23a8734b86b7b2b98263fe2e7f [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:42.275285 27125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:42.276249 27125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:42.276356 27125 catalog_manager.cc:1259] Loaded cluster ID: 2f8b7cedff814801aee6d23b2b5583e8
I20260927 18:41:42.276588 27125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260927 18:41:42.276857 27125 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f00699369ad google::LogMessage::Fail() at ??:0
@ 0x7f006993988a google::LogMessage::SendToLog() at ??:0
@ 0x7f00699364de google::LogMessage::Flush() at ??:0
@ 0x7f0069937a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f00718570f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f00718608b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f006b8040d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f006b7f9954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f006c65d6db start_thread at ??:0
@ 0x7f006843461f clone at ??:0
W20260927 18:41:42.374070 27119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:42.374339 27119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:42.374382 27119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:42.375603 27119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:42.375680 27119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:42.375732 27119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:41:42.377316 27119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:43337
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=35185
--tserver_master_addrs=127.11.251.254:42519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:42.377673 27119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:42.377947 27119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:42.383420 27154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:42.383728 27153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:42.383868 27156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:42.383931 27119 server_base.cc:1034] running on GCE node
I20260927 18:41:42.384198 27119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:42.384511 27119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:42.385654 27119 hybrid_clock.cc:648] HybridClock initialized: now 1790534502385634 us; error 43 us; skew 500 ppm
Sep 27 18:41:42 dist-test-slave-z47j krb5kdc[25979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534502, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:42.389081 27119 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:41:42.389640 27119 webserver.cc:460] Webserver started at http://127.11.251.193:35185/ using document root <none> and password file <none>
I20260927 18:41:42.389931 27119 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:42.389981 27119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:42.391543 27119 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:41:42.392488 27163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:42.392717 27119 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:41:42.392864 27119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "057edf154c3a45a2bfc592ef9e5bce20"
format_stamp: "Formatted at 2026-09-27 18:41:40 on dist-test-slave-z47j"
server_key: "3709558c78db446ea4dacf808da7c95e"
server_key_iv: "5d85c5cbbe6d07dcffa5bca48e0968e5"
server_key_version: "encryptionkey@0"
I20260927 18:41:42.393308 27119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:42.404708 27119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:42.407582 27119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:42.407759 27119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:42.408016 27119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:42.408545 27170 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 18:41:42.410737 27119 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 18:41:42.410848 27119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260927 18:41:42.410900 27119 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 18:41:42.411651 27119 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 18:41:42.411710 27119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260927 18:41:42.411814 27170 tablet_bootstrap.cc:492] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20: Bootstrap starting.
I20260927 18:41:42.414040 27170 log.cc:824] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:42.418108 27170 tablet_bootstrap.cc:492] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:41:42.418490 27170 tablet_bootstrap.cc:492] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20: Bootstrap complete.
I20260927 18:41:42.418689 27170 ts_tablet_manager.cc:1397] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260927 18:41:42.420258 27119 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:43337
I20260927 18:41:42.420360 27278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:43337 every 8 connection(s)
I20260927 18:41:42.420677 27119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:41:42.421226 27170 raft_consensus.cc:348] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } }
I20260927 18:41:42.421686 27170 raft_consensus.cc:729] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 057edf154c3a45a2bfc592ef9e5bce20, State: Initialized, Role: FOLLOWER
I20260927 18:41:42.421823 27170 consensus_queue.cc:260] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } }
I20260927 18:41:42.422111 27170 ts_tablet_manager.cc:1428] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20260927 18:41:42.430783 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 27119
I20260927 18:41:42.431077 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:37107
--local_ip_for_outbound_sockets=127.11.251.194
--tserver_master_addrs=127.11.251.254:42519
--webserver_port=40725
--webserver_interface=127.11.251.194
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:42.526535 27281 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:41:42.421551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.193:43023)
0927 18:41:42.421788 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:42.421838 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:42.422029 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:42.526274 (+104245us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":150,"threads_started":1}
W20260927 18:41:42.526937 27279 heartbeater.cc:644] Failed to heartbeat to 127.11.251.254:42519 (0 consecutive failures): Network error: Failed to ping master at 127.11.251.254:42519: Client connection negotiation failed: client connection to 127.11.251.254:42519: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 18:41:42.527372 27281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:42.527220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.193:60091)
0927 18:41:42.527288 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:42.527341 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260927 18:41:42.527736 27281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:42.527637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.193:49915)
0927 18:41:42.527685 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:42.527716 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260927 18:41:42.527863 27279 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 18:41:42.540421 27283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:42.540642 27283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:42.540666 27283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:42.542063 27283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:42.542115 27283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:42.542143 27283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:41:42.543730 27283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:37107
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=40725
--tserver_master_addrs=127.11.251.254:42519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:42.544026 27283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:42.544299 27283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:42.549865 27291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:42.549875 27288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:42.549916 27289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:42.550175 27283 server_base.cc:1034] running on GCE node
I20260927 18:41:42.550386 27283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:42.550629 27283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:42.551829 27283 hybrid_clock.cc:648] HybridClock initialized: now 1790534502551812 us; error 28 us; skew 500 ppm
Sep 27 18:41:42 dist-test-slave-z47j krb5kdc[25979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534502, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:42.554801 27283 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:41:42.555294 27283 webserver.cc:460] Webserver started at http://127.11.251.194:40725/ using document root <none> and password file <none>
I20260927 18:41:42.555560 27283 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:42.555635 27283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:42.557083 27283 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 18:41:42.557829 27298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:42.558043 27283 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:41:42.558164 27283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "8243ccc2001c4efe8144c247db62a157"
format_stamp: "Formatted at 2026-09-27 18:41:40 on dist-test-slave-z47j"
server_key: "9453a5bf33e1c67688412a08c751e9e6"
server_key_iv: "e94efb69fa4c6ae49f14e2b9a8281543"
server_key_version: "encryptionkey@0"
I20260927 18:41:42.558614 27283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:42.574712 27283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:42.577068 27283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:42.577260 27283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:42.577543 27283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:42.577934 27283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:42.577998 27283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:42.578049 27283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:42.578099 27283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:42.585726 27283 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:37107
I20260927 18:41:42.585811 27411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:37107 every 8 connection(s)
I20260927 18:41:42.586252 27283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:41:42.587947 27414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:42.587201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.194:55275)
0927 18:41:42.587651 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:42.587775 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":81,"threads_started":1}
W20260927 18:41:42.588577 27412 heartbeater.cc:644] Failed to heartbeat to 127.11.251.254:42519 (0 consecutive failures): Network error: Failed to ping master at 127.11.251.254:42519: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
I20260927 18:41:42.588786 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 27283
I20260927 18:41:42.589116 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:35517
--local_ip_for_outbound_sockets=127.11.251.195
--tserver_master_addrs=127.11.251.254:42519
--webserver_port=45865
--webserver_interface=127.11.251.195
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:41:42.589464 27414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:42.589312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.194:36503)
0927 18:41:42.589375 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:42.589418 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 18:41:42.590104 27414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:42.590007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.194:36831)
0927 18:41:42.590053 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:42.590076 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260927 18:41:42.590389 27412 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 18:41:42.685062 27415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:42.685328 27415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:42.685357 27415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:42.686666 27415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:42.686705 27415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:42.686733 27415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:41:42.688306 27415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:35517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=45865
--tserver_master_addrs=127.11.251.254:42519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:42.688545 27415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:42.688781 27415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:42.694701 27421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:42.694707 27423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:42.694739 27420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:42.695103 27415 server_base.cc:1034] running on GCE node
I20260927 18:41:42.695344 27415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:42.695675 27415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:42.696846 27415 hybrid_clock.cc:648] HybridClock initialized: now 1790534502696834 us; error 38 us; skew 500 ppm
Sep 27 18:41:42 dist-test-slave-z47j krb5kdc[25979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534502, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:42.699623 27415 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:41:42.700124 27415 webserver.cc:460] Webserver started at http://127.11.251.195:45865/ using document root <none> and password file <none>
I20260927 18:41:42.700387 27415 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:42.700435 27415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:42.702023 27415 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 18:41:42.702699 27430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:42.702903 27415 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:41:42.702998 27415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "e4538361d248498bab4699b9d945e7cc"
format_stamp: "Formatted at 2026-09-27 18:41:41 on dist-test-slave-z47j"
server_key: "f890584ae50388a89d665b9993a4c687"
server_key_iv: "e323d53d0b4869a4ca05c8ae4d82d737"
server_key_version: "encryptionkey@0"
I20260927 18:41:42.703428 27415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:42.720491 27415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:42.723022 27415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:42.723192 27415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:42.723513 27415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:42.723862 27415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:42.723898 27415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:42.723923 27415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:42.723968 27415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:42.730953 27415 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:35517
I20260927 18:41:42.731036 27543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:35517 every 8 connection(s)
I20260927 18:41:42.731437 27415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:41:42.732649 27546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:42.732226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.195:45279)
0927 18:41:42.732395 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:42.732484 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
W20260927 18:41:42.733203 27544 heartbeater.cc:644] Failed to heartbeat to 127.11.251.254:42519 (0 consecutive failures): Network error: Failed to ping master at 127.11.251.254:42519: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
I20260927 18:41:42.733518 27546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:42.733421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.195:53161)
0927 18:41:42.733469 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:42.733498 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260927 18:41:42.734071 27546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:42.733974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.195:53475)
0927 18:41:42.734021 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:42.734055 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260927 18:41:42.734196 27544 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 18:41:42.736716 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 27415
I20260927 18:41:43.528649 27548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:43.528311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.193:33027)
0927 18:41:43.528492 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:43.528574 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":81,"threads_started":1}
I20260927 18:41:43.574633 27549 raft_consensus.cc:482] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:41:43.574838 27549 raft_consensus.cc:504] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } }
I20260927 18:41:43.575271 27549 leader_election.cc:288] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8243ccc2001c4efe8144c247db62a157 (127.11.251.194:37107), e4538361d248498bab4699b9d945e7cc (127.11.251.195:35517)
Sep 27 18:41:43 dist-test-slave-z47j krb5kdc[25979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534502, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.194@KRBTEST.COM
Sep 27 18:41:43 dist-test-slave-z47j krb5kdc[25979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534502, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.195@KRBTEST.COM
I20260927 18:41:43.589826 27551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:43.575632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:45683 (local address 127.11.251.194:37107)
0927 18:41:43.575953 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:43.575970 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:43.576012 (+ 42us) server_negotiation.cc:407] Connection header received
0927 18:41:43.576668 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:43.576671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:43.576754 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:43.576922 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:43.578229 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:43.579807 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:43.581546 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:43.582213 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:43.582343 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:43.585873 (+ 3530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:43.585879 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:43.585879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:43.585893 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:43.587988 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:43.588500 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:43.588503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:43.588504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:43.588592 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:43.588866 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:43.588867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:43.588867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:43.589127 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:43.589227 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:43.589533 (+ 306us) server_negotiation.cc:299] Negotiation successful
0927 18:41:43.589705 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":108,"threads_started":1}
I20260927 18:41:43.589835 27552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:43.575741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:60747 (local address 127.11.251.195:35517)
0927 18:41:43.575990 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:43.575998 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:43.576037 (+ 39us) server_negotiation.cc:407] Connection header received
0927 18:41:43.576668 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:43.576671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:43.576758 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:43.576922 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:43.578402 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:43.579807 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:43.581546 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:43.582213 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:43.582341 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:43.586070 (+ 3729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:43.586074 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:43.586074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:43.586086 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:43.588073 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:43.588582 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:43.588585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:43.588586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:43.588684 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:43.588897 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:43.588899 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:43.588899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:43.589157 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:43.589256 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:43.589537 (+ 281us) server_negotiation.cc:299] Negotiation successful
0927 18:41:43.589700 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":67,"threads_started":1}
I20260927 18:41:43.589834 27548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:43.575449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:37107 (local address 127.11.251.193:45683)
0927 18:41:43.575572 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:43.575931 (+ 359us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:43.576023 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:43.577014 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:43.577018 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:43.577617 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:43.578059 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:43.578074 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:43.579982 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:43.579982 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:43.581436 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:43.581439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:43.582328 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:43.582329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:43.582515 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:43.583275 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:43.583280 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:43.585713 (+ 2433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:43.588091 (+ 2378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:43.588098 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:43.588101 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:43.588392 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:43.588666 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:43.588667 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:43.588668 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:43.588792 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:43.589204 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:43.589211 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:43.589362 (+ 151us) client_negotiation.cc:769] Sending connection context
0927 18:41:43.589481 (+ 119us) client_negotiation.cc:240] Negotiation successful
0927 18:41:43.589731 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":152}
I20260927 18:41:43.590129 27550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:43.575573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:35517 (local address 127.11.251.193:60747)
0927 18:41:43.575736 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:43.575749 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:43.575818 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:43.577014 (+ 1196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:43.577018 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:43.577617 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:43.578266 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:43.578269 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:43.579967 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:43.579969 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:43.581436 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:43.581439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:43.582326 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:43.582327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:43.582515 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:43.583242 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:43.583254 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:43.585953 (+ 2699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:43.588172 (+ 2219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:43.588173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:43.588174 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:43.588482 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:43.588759 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:43.588761 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:43.588761 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:43.588833 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:43.589236 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:43.589237 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:43.589362 (+ 125us) client_negotiation.cc:769] Sending connection context
0927 18:41:43.589481 (+ 119us) client_negotiation.cc:240] Negotiation successful
0927 18:41:43.589831 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":231,"thread_start_us":62,"threads_started":1}
I20260927 18:41:43.591043 27553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:43.590722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.194:40891)
0927 18:41:43.590889 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:43.590956 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":77,"threads_started":1}
W20260927 18:41:43.591046 27164 leader_election.cc:341] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e4538361d248498bab4699b9d945e7cc (127.11.251.195:35517): Not found: Tablet not found: f2441f107302416085f4eca7a0e8c11a
W20260927 18:41:43.591408 27164 leader_election.cc:341] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8243ccc2001c4efe8144c247db62a157 (127.11.251.194:37107): Not found: Tablet not found: f2441f107302416085f4eca7a0e8c11a
I20260927 18:41:43.591483 27164 leader_election.cc:302] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 057edf154c3a45a2bfc592ef9e5bce20; no voters: 8243ccc2001c4efe8144c247db62a157, e4538361d248498bab4699b9d945e7cc
I20260927 18:41:43.591634 27549 raft_consensus.cc:2726] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 18:41:43.734809 27554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:43.734559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.195:41019)
0927 18:41:43.734706 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:43.734759 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":66,"threads_started":1}
I20260927 18:41:44.529640 27555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:44.529291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.193:32833)
0927 18:41:44.529488 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:44.529529 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":110,"threads_started":1}
I20260927 18:41:44.591765 27556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:44.591494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.194:41517)
0927 18:41:44.591646 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:44.591707 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":66,"threads_started":1}
I20260927 18:41:44.735852 27557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:44.735598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.195:39157)
0927 18:41:44.735768 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:44.735808 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":85,"threads_started":1}
I20260927 18:41:45.252432 27558 raft_consensus.cc:482] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:41:45.252539 27558 raft_consensus.cc:504] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } }
I20260927 18:41:45.252791 27558 leader_election.cc:288] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8243ccc2001c4efe8144c247db62a157 (127.11.251.194:37107), e4538361d248498bab4699b9d945e7cc (127.11.251.195:35517)
W20260927 18:41:45.253508 27164 leader_election.cc:341] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8243ccc2001c4efe8144c247db62a157 (127.11.251.194:37107): Not found: Tablet not found: f2441f107302416085f4eca7a0e8c11a
W20260927 18:41:45.253676 27164 leader_election.cc:341] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e4538361d248498bab4699b9d945e7cc (127.11.251.195:35517): Not found: Tablet not found: f2441f107302416085f4eca7a0e8c11a
I20260927 18:41:45.253715 27164 leader_election.cc:302] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 057edf154c3a45a2bfc592ef9e5bce20; no voters: 8243ccc2001c4efe8144c247db62a157, e4538361d248498bab4699b9d945e7cc
I20260927 18:41:45.253823 27558 raft_consensus.cc:2726] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 18:41:45.530588 27559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:45.530290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.193:59283)
0927 18:41:45.530439 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:45.530499 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":66,"threads_started":1}
I20260927 18:41:45.592597 27560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:45.592288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.194:51527)
0927 18:41:45.592462 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:45.592532 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":69,"threads_started":1}
I20260927 18:41:45.736779 27561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:45.736471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.195:33843)
0927 18:41:45.736673 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:45.736723 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":104,"threads_started":1}
I20260927 18:41:46.531488 27562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:46.531197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.193:34169)
0927 18:41:46.531391 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:46.531432 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":86,"threads_started":1}
I20260927 18:41:46.593530 27563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:46.593269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.194:56719)
0927 18:41:46.593425 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:46.593473 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":73,"threads_started":1}
I20260927 18:41:46.737758 27564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:46.737406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.195:43267)
0927 18:41:46.737655 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:46.737703 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":103,"threads_started":1}
I20260927 18:41:46.805485 27565 raft_consensus.cc:482] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:41:46.805734 27565 raft_consensus.cc:504] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } }
I20260927 18:41:46.805995 27565 leader_election.cc:288] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8243ccc2001c4efe8144c247db62a157 (127.11.251.194:37107), e4538361d248498bab4699b9d945e7cc (127.11.251.195:35517)
W20260927 18:41:46.806636 27164 leader_election.cc:341] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8243ccc2001c4efe8144c247db62a157 (127.11.251.194:37107): Not found: Tablet not found: f2441f107302416085f4eca7a0e8c11a
W20260927 18:41:46.806722 27164 leader_election.cc:341] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e4538361d248498bab4699b9d945e7cc (127.11.251.195:35517): Not found: Tablet not found: f2441f107302416085f4eca7a0e8c11a
I20260927 18:41:46.806759 27164 leader_election.cc:302] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 057edf154c3a45a2bfc592ef9e5bce20; no voters: 8243ccc2001c4efe8144c247db62a157, e4538361d248498bab4699b9d945e7cc
I20260927 18:41:46.806880 27565 raft_consensus.cc:2726] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 18:41:47.532634 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:47.532290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.193:58815)
0927 18:41:47.532515 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:47.532572 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":122,"threads_started":1}
I20260927 18:41:47.594513 27567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:47.594170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.194:60963)
0927 18:41:47.594367 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:47.594410 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":88,"threads_started":1}
I20260927 18:41:47.738657 27568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:47.738327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.195:58579)
0927 18:41:47.738546 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:47.738602 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":133,"threads_started":1}
I20260927 18:41:48.533532 27569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:48.533193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.193:51483)
0927 18:41:48.533399 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:48.533449 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":73,"threads_started":1}
I20260927 18:41:48.595434 27570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:48.595149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.194:49519)
0927 18:41:48.595335 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:48.595385 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":108,"threads_started":1}
I20260927 18:41:48.739553 27571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:48.739222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.195:56903)
0927 18:41:48.739425 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:48.739490 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":88,"threads_started":1}
I20260927 18:41:49.534549 27572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:49.534242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.193:60837)
0927 18:41:49.534423 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:49.534484 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":73,"threads_started":1}
I20260927 18:41:49.596264 27573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:49.595984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.194:60335)
0927 18:41:49.596143 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:49.596202 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":70,"threads_started":1}
I20260927 18:41:49.740532 27574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:49.740216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.195:53483)
0927 18:41:49.740411 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:49.740475 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":85,"threads_started":1}
I20260927 18:41:50.535557 27575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:50.535232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.193:46657)
0927 18:41:50.535436 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:50.535478 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":95,"threads_started":1}
I20260927 18:41:50.597131 27576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:50.596828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.194:50809)
0927 18:41:50.597031 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:50.597083 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":122,"threads_started":1}
I20260927 18:41:50.741420 27577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:50.741108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.195:40873)
0927 18:41:50.741294 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:50.741356 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":75,"threads_started":1}
I20260927 18:41:51.041245 27578 raft_consensus.cc:482] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:41:51.041352 27578 raft_consensus.cc:504] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } }
I20260927 18:41:51.041520 27578 leader_election.cc:288] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8243ccc2001c4efe8144c247db62a157 (127.11.251.194:37107), e4538361d248498bab4699b9d945e7cc (127.11.251.195:35517)
W20260927 18:41:51.042073 27164 leader_election.cc:341] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8243ccc2001c4efe8144c247db62a157 (127.11.251.194:37107): Not found: Tablet not found: f2441f107302416085f4eca7a0e8c11a
W20260927 18:41:51.042263 27164 leader_election.cc:341] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e4538361d248498bab4699b9d945e7cc (127.11.251.195:35517): Not found: Tablet not found: f2441f107302416085f4eca7a0e8c11a
I20260927 18:41:51.042351 27164 leader_election.cc:302] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 057edf154c3a45a2bfc592ef9e5bce20; no voters: 8243ccc2001c4efe8144c247db62a157, e4538361d248498bab4699b9d945e7cc
I20260927 18:41:51.042472 27578 raft_consensus.cc:2726] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 18:41:51.536559 27579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:51.536256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.193:42257)
0927 18:41:51.536445 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:51.536503 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":85,"threads_started":1}
I20260927 18:41:51.598023 27580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:51.597705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.194:51813)
0927 18:41:51.597871 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:51.597930 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":69,"threads_started":1}
I20260927 18:41:51.742304 27581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:51.742025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.195:60373)
0927 18:41:51.742176 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:51.742241 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":69,"threads_started":1}
I20260927 18:41:52.537406 27582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:52.537075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.193:34879)
0927 18:41:52.537289 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:52.537352 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":100,"threads_started":1}
I20260927 18:41:52.598990 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:52.598652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.194:46995)
0927 18:41:52.598860 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:52.598941 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":85,"threads_started":1}
I20260927 18:41:52.743192 27584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:52.742912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.195:43605)
0927 18:41:52.743076 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:52.743130 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":68,"threads_started":1}
I20260927 18:41:53.538389 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:53.538107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.193:38173)
0927 18:41:53.538289 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:53.538334 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":85,"threads_started":1}
I20260927 18:41:53.599856 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:53.599552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.194:60031)
0927 18:41:53.599729 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:53.599792 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":71,"threads_started":1}
I20260927 18:41:53.744199 27587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:53.743877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.195:47781)
0927 18:41:53.744070 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:53.744113 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":83,"threads_started":1}
I20260927 18:41:54.539507 27588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:54.539132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.193:41257)
0927 18:41:54.539393 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:54.539453 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":175,"threads_started":1}
I20260927 18:41:54.600813 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:54.600479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.194:39567)
0927 18:41:54.600690 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:54.600751 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":72,"threads_started":1}
I20260927 18:41:54.745015 27590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:54.744708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.195:33719)
0927 18:41:54.744884 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:54.744940 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":70,"threads_started":1}
I20260927 18:41:55.540596 27591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:55.540205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.193:49419)
0927 18:41:55.540439 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:55.540503 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":87,"threads_started":1}
I20260927 18:41:55.601830 27592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:55.601460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.194:41933)
0927 18:41:55.601708 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:55.601760 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":83,"threads_started":1}
I20260927 18:41:55.746008 27593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:55.745701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.195:35947)
0927 18:41:55.745895 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:55.745958 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":87,"threads_started":1}
I20260927 18:41:56.139017 27594 raft_consensus.cc:482] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:41:56.139192 27594 raft_consensus.cc:504] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8243ccc2001c4efe8144c247db62a157" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 37107 } } peers { permanent_uuid: "e4538361d248498bab4699b9d945e7cc" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 35517 } } peers { permanent_uuid: "057edf154c3a45a2bfc592ef9e5bce20" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43337 } }
I20260927 18:41:56.139451 27594 leader_election.cc:288] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8243ccc2001c4efe8144c247db62a157 (127.11.251.194:37107), e4538361d248498bab4699b9d945e7cc (127.11.251.195:35517)
W20260927 18:41:56.139974 27164 leader_election.cc:341] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8243ccc2001c4efe8144c247db62a157 (127.11.251.194:37107): Not found: Tablet not found: f2441f107302416085f4eca7a0e8c11a
W20260927 18:41:56.140100 27164 leader_election.cc:341] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e4538361d248498bab4699b9d945e7cc (127.11.251.195:35517): Not found: Tablet not found: f2441f107302416085f4eca7a0e8c11a
I20260927 18:41:56.140120 27164 leader_election.cc:302] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 057edf154c3a45a2bfc592ef9e5bce20; no voters: 8243ccc2001c4efe8144c247db62a157, e4538361d248498bab4699b9d945e7cc
I20260927 18:41:56.140208 27594 raft_consensus.cc:2726] T f2441f107302416085f4eca7a0e8c11a P 057edf154c3a45a2bfc592ef9e5bce20 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 18:41:56.541522 27595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:56.541195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.193:53859)
0927 18:41:56.541406 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:56.541468 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":93,"threads_started":1}
I20260927 18:41:56.602807 27596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:56.602521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.194:38587)
0927 18:41:56.602686 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:56.602738 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":73,"threads_started":1}
I20260927 18:41:56.746898 27597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:56.746598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.195:45619)
0927 18:41:56.746789 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:56.746850 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":87,"threads_started":1}
I20260927 18:41:57.542618 27598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:57.542264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.193:36113)
0927 18:41:57.542467 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:57.542567 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":81,"threads_started":1}
I20260927 18:41:57.603808 27599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:57.603552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.194:34721)
0927 18:41:57.603712 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:57.603760 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":82,"threads_started":1}
I20260927 18:41:57.746447 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 27119
I20260927 18:41:57.748739 27600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:57.748056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:42519 (local address 127.11.251.195:44125)
0927 18:41:57.748644 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:57.748685 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:42519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":72,"threads_started":1}
I20260927 18:41:57.753355 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 27283
I20260927 18:41:57.759392 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 27415
2026-09-27T18:41:57Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18574 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:57 dist-test-slave-z47j krb5kdc[27603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:57 dist-test-slave-z47j krb5kdc[27603](info): set up 2 sockets
Sep 27 18:41:57 dist-test-slave-z47j krb5kdc[27603](info): commencing operation
krb5kdc: starting...
W20260927 18:41:58.854151 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:41:58 dist-test-slave-z47j krb5kdc[27603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:41:58.981050 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:41973
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:41973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:59.074123 27616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:59.074362 27616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:59.074384 27616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:59.075470 27616 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 18:41:59.075505 27616 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 18:41:59.075518 27616 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 18:41:59.075536 27616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:59.075549 27616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:59.075562 27616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:59.075574 27616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:59.075587 27616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:59.077420 27616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:41973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:41973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:59.077844 27616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:59.078071 27616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:59.083922 27624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:59.083947 27622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:59.084000 27616 server_base.cc:1034] running on GCE node
W20260927 18:41:59.084002 27621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:59.084591 27616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:59.084934 27616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:59.086084 27616 hybrid_clock.cc:648] HybridClock initialized: now 1790534519086067 us; error 39 us; skew 500 ppm
Sep 27 18:41:59 dist-test-slave-z47j krb5kdc[27603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534519, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:59.088876 27616 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:41:59.089362 27616 webserver.cc:460] Webserver started at http://127.11.251.254:43491/ using document root <none> and password file <none>
I20260927 18:41:59.089666 27616 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:59.089722 27616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:59.089865 27616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:59.091048 27616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "af6f2bd605f1446697dd1ecda55f6393"
format_stamp: "Formatted at 2026-09-27 18:41:59 on dist-test-slave-z47j"
server_key: "804a4af3a7f8c4ccfcbd09a4c41a15c5"
server_key_iv: "26c41aa48398c945e26e2a78f2e94950"
server_key_version: "encryptionkey@0"
I20260927 18:41:59.091486 27616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "af6f2bd605f1446697dd1ecda55f6393"
format_stamp: "Formatted at 2026-09-27 18:41:59 on dist-test-slave-z47j"
server_key: "804a4af3a7f8c4ccfcbd09a4c41a15c5"
server_key_iv: "26c41aa48398c945e26e2a78f2e94950"
server_key_version: "encryptionkey@0"
I20260927 18:41:59.092990 27616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:41:59.093937 27631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:59.094234 27616 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:41:59.094326 27616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "af6f2bd605f1446697dd1ecda55f6393"
format_stamp: "Formatted at 2026-09-27 18:41:59 on dist-test-slave-z47j"
server_key: "804a4af3a7f8c4ccfcbd09a4c41a15c5"
server_key_iv: "26c41aa48398c945e26e2a78f2e94950"
server_key_version: "encryptionkey@0"
I20260927 18:41:59.094461 27616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:59.125381 27616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:59.127859 27616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:59.128005 27616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:59.132688 27616 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:41973
I20260927 18:41:59.132900 27683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:41973 every 8 connection(s)
I20260927 18:41:59.133131 27616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:59.134245 27684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:59.136924 27684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393: Bootstrap starting.
I20260927 18:41:59.137528 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 27616
I20260927 18:41:59.137681 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:41:59.137862 27684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:59.137897 12271 external_mini_cluster.cc:1428] Setting key aa6060d98dd2eee6d697238eee303fef
I20260927 18:41:59.138237 27684 log.cc:824] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:59.139138 27684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393: No bootstrap required, opened a new log
I20260927 18:41:59.141860 27684 raft_consensus.cc:348] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6f2bd605f1446697dd1ecda55f6393" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41973 } }
I20260927 18:41:59.142058 27684 raft_consensus.cc:374] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:59.142091 27684 raft_consensus.cc:729] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af6f2bd605f1446697dd1ecda55f6393, State: Initialized, Role: FOLLOWER
I20260927 18:41:59.142222 27684 consensus_queue.cc:260] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6f2bd605f1446697dd1ecda55f6393" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41973 } }
I20260927 18:41:59.142321 27684 raft_consensus.cc:388] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:59.142372 27684 raft_consensus.cc:482] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:59.142431 27684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:59.143239 27684 raft_consensus.cc:504] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6f2bd605f1446697dd1ecda55f6393" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41973 } }
I20260927 18:41:59.143338 27684 leader_election.cc:302] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af6f2bd605f1446697dd1ecda55f6393; no voters:
I20260927 18:41:59.143528 27684 leader_election.cc:288] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:59.143638 27689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:59.143826 27684 sys_catalog.cc:564] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:59.143898 27689 raft_consensus.cc:686] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 1 LEADER]: Becoming Leader. State: Replica: af6f2bd605f1446697dd1ecda55f6393, State: Running, Role: LEADER
I20260927 18:41:59.144089 27689 consensus_queue.cc:237] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6f2bd605f1446697dd1ecda55f6393" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41973 } }
I20260927 18:41:59.144606 27691 sys_catalog.cc:455] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af6f2bd605f1446697dd1ecda55f6393. Latest consensus state: current_term: 1 leader_uuid: "af6f2bd605f1446697dd1ecda55f6393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6f2bd605f1446697dd1ecda55f6393" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41973 } } }
I20260927 18:41:59.144788 27691 sys_catalog.cc:458] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:59.144820 27690 sys_catalog.cc:455] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af6f2bd605f1446697dd1ecda55f6393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6f2bd605f1446697dd1ecda55f6393" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41973 } } }
I20260927 18:41:59.144912 27690 sys_catalog.cc:458] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:59.145126 27698 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 27 18:41:59 dist-test-slave-z47j krb5kdc[27603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:41:59.145781 27698 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:59.148011 27698 catalog_manager.cc:1347] Generated new cluster ID: e2cee189163347e1b3cf43ccdfa85f11
I20260927 18:41:59.148065 27698 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:59.150374 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:59.138409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51268 (local address 127.11.251.254:41973)
0927 18:41:59.138607 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:59.138616 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:59.138646 (+ 30us) server_negotiation.cc:407] Connection header received
0927 18:41:59.139110 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:59.139117 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:59.139168 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:59.139362 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:59.140164 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.141156 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.142295 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.142847 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.142943 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:59.146320 (+ 3377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:59.146325 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:59.146325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:59.146339 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:59.148547 (+ 2208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:59.149032 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:59.149034 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:59.149035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:59.149145 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:59.149378 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:59.149378 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:59.149379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:59.149717 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:59.149852 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:59.150006 (+ 154us) server_negotiation.cc:299] Negotiation successful
0927 18:41:59.150148 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":79,"threads_started":1}
I20260927 18:41:59.164913 27698 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:59.165807 27698 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:59.174360 27698 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393: Generated new TSK 0
I20260927 18:41:59.174643 27698 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.251.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
I20260927 18:41:59.209949 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41973
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:59.301934 27710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:59.302210 27710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:59.302250 27710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:59.303445 27710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:59.303512 27710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:59.303560 27710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:41:59.305084 27710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:59.305377 27710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:59.305656 27710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:59.311364 27715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:59.311334 27716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:59.311328 27718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:59.311661 27710 server_base.cc:1034] running on GCE node
I20260927 18:41:59.311911 27710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:59.312204 27710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:59.313366 27710 hybrid_clock.cc:648] HybridClock initialized: now 1790534519313354 us; error 48 us; skew 500 ppm
Sep 27 18:41:59 dist-test-slave-z47j krb5kdc[27603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534519, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:59.316161 27710 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:41:59.316720 27710 webserver.cc:460] Webserver started at http://127.11.251.193:37071/ using document root <none> and password file <none>
I20260927 18:41:59.316965 27710 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:59.317035 27710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:59.317183 27710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:59.318336 27710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "3e5984283f154ce896d6ade588666522"
format_stamp: "Formatted at 2026-09-27 18:41:59 on dist-test-slave-z47j"
server_key: "73443c1737cfb89734313f03b7891039"
server_key_iv: "251295dc151a5b5fa3248faca4aa16f3"
server_key_version: "encryptionkey@0"
I20260927 18:41:59.318702 27710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "3e5984283f154ce896d6ade588666522"
format_stamp: "Formatted at 2026-09-27 18:41:59 on dist-test-slave-z47j"
server_key: "73443c1737cfb89734313f03b7891039"
server_key_iv: "251295dc151a5b5fa3248faca4aa16f3"
server_key_version: "encryptionkey@0"
I20260927 18:41:59.320191 27710 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260927 18:41:59.321094 27725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:59.321298 27710 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:41:59.321409 27710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "3e5984283f154ce896d6ade588666522"
format_stamp: "Formatted at 2026-09-27 18:41:59 on dist-test-slave-z47j"
server_key: "73443c1737cfb89734313f03b7891039"
server_key_iv: "251295dc151a5b5fa3248faca4aa16f3"
server_key_version: "encryptionkey@0"
I20260927 18:41:59.321498 27710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:59.329658 27710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:59.331756 27710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:59.331911 27710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:59.332172 27710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:59.332494 27710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:59.332546 27710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:59.332582 27710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:59.332624 27710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:59.339740 27710 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:46563
I20260927 18:41:59.339816 27838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:46563 every 8 connection(s)
I20260927 18:41:59.340154 27710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:41:59.345957 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 27710
I20260927 18:41:59.346087 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:41:59.346279 12271 external_mini_cluster.cc:1428] Setting key 596e163d1de592bd1e1b15299da33a13
Sep 27 18:41:59 dist-test-slave-z47j krb5kdc[27603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534519, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:41:59.351428 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:59.341142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:33933 (local address 127.11.251.254:41973)
0927 18:41:59.341242 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:59.341246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:59.341437 (+ 191us) server_negotiation.cc:407] Connection header received
0927 18:41:59.342210 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:59.342212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:59.342231 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:59.342298 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:59.343431 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.344086 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.345302 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.345517 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.345613 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:59.348373 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:59.348376 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:59.348376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:59.348384 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:59.349924 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:59.350351 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:59.350352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:59.350352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:59.350413 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:59.350695 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:59.350696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:59.350697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:59.350838 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:59.350910 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:59.351210 (+ 300us) server_negotiation.cc:299] Negotiation successful
0927 18:41:59.351322 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 18:41:59.351644 27841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:59.341041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:33933)
0927 18:41:59.341274 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:59.341334 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:59.342037 (+ 703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:59.342391 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:59.342396 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:59.343072 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:59.343319 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:59.343325 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.344189 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.344191 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:59.345201 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:59.345203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.345630 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.345631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:59.345763 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:59.346481 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:59.346494 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:59.348273 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:59.350019 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:59.350027 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:59.350030 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:59.350272 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:59.350478 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:59.350479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:59.350479 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:59.350638 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:59.350911 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:59.350920 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:59.351071 (+ 151us) client_negotiation.cc:769] Sending connection context
0927 18:41:59.351180 (+ 109us) client_negotiation.cc:240] Negotiation successful
0927 18:41:59.351382 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":112,"threads_started":1}
I20260927 18:41:59.352154 27839 heartbeater.cc:347] Connected to a master server at 127.11.251.254:41973
I20260927 18:41:59.352259 27839 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:59.352491 27839 heartbeater.cc:511] Master 127.11.251.254:41973 requested a full tablet report, sending...
I20260927 18:41:59.353096 27648 ts_manager.cc:194] Registered new tserver with Master: 3e5984283f154ce896d6ade588666522 (127.11.251.193:46563)
I20260927 18:41:59.353847 27648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:33933
WARNING: no policy specified for kudu/127.11.251.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
I20260927 18:41:59.377328 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41973
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:59.472889 27844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:59.473114 27844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:59.473137 27844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:59.474328 27844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:59.474371 27844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:59.474398 27844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:41:59.475838 27844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:59.476051 27844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:59.476240 27844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:59.481541 27852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:59.481771 27844 server_base.cc:1034] running on GCE node
W20260927 18:41:59.481541 27850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:59.481534 27849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:59.482067 27844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:59.482323 27844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:59.483484 27844 hybrid_clock.cc:648] HybridClock initialized: now 1790534519483475 us; error 35 us; skew 500 ppm
Sep 27 18:41:59 dist-test-slave-z47j krb5kdc[27603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534519, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:59.486166 27844 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:41:59.486575 27844 webserver.cc:460] Webserver started at http://127.11.251.194:33107/ using document root <none> and password file <none>
I20260927 18:41:59.486771 27844 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:59.486820 27844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:59.486919 27844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:59.487977 27844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "1d2806f483c04b6592f08e3bb1f7f62d"
format_stamp: "Formatted at 2026-09-27 18:41:59 on dist-test-slave-z47j"
server_key: "9e43ea0d5ef162db2b81f9a7786b70c0"
server_key_iv: "0402299b95d605748accd4ebf664b60b"
server_key_version: "encryptionkey@0"
I20260927 18:41:59.488296 27844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "1d2806f483c04b6592f08e3bb1f7f62d"
format_stamp: "Formatted at 2026-09-27 18:41:59 on dist-test-slave-z47j"
server_key: "9e43ea0d5ef162db2b81f9a7786b70c0"
server_key_iv: "0402299b95d605748accd4ebf664b60b"
server_key_version: "encryptionkey@0"
I20260927 18:41:59.489638 27844 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260927 18:41:59.490408 27859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:59.490563 27844 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:41:59.490653 27844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "1d2806f483c04b6592f08e3bb1f7f62d"
format_stamp: "Formatted at 2026-09-27 18:41:59 on dist-test-slave-z47j"
server_key: "9e43ea0d5ef162db2b81f9a7786b70c0"
server_key_iv: "0402299b95d605748accd4ebf664b60b"
server_key_version: "encryptionkey@0"
I20260927 18:41:59.490711 27844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:59.519032 27844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:59.521652 27844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:59.521799 27844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:59.522042 27844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:59.522408 27844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:59.522444 27844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:59.522498 27844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:59.522533 27844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:59.529536 27844 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:34265
I20260927 18:41:59.529655 27972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:34265 every 8 connection(s)
I20260927 18:41:59.530025 27844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:41:59.534307 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 27844
I20260927 18:41:59.534438 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:41:59.534646 12271 external_mini_cluster.cc:1428] Setting key b469c02774db48f101abd38d52415aea
Sep 27 18:41:59 dist-test-slave-z47j krb5kdc[27603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534519, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:41:59.541122 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:59.531213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:46475 (local address 127.11.251.254:41973)
0927 18:41:59.531313 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:59.531316 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:59.531576 (+ 260us) server_negotiation.cc:407] Connection header received
0927 18:41:59.532212 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:59.532214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:59.532226 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:59.532287 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:59.533323 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.533922 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.535311 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.535546 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.535620 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:59.538009 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:59.538011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:59.538012 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:59.538020 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:59.539511 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:59.540018 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:59.540019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:59.540020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:59.540079 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:59.540407 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:59.540408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:59.540409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:59.540529 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:59.540590 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:59.540971 (+ 381us) server_negotiation.cc:299] Negotiation successful
0927 18:41:59.541045 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 18:41:59.541461 27975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:59.531167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:46475)
0927 18:41:59.531435 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:59.531478 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:59.532083 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:59.532349 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:59.532353 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:59.532986 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:59.533235 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:59.533239 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.534021 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.534022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:59.535167 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:59.535170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.535641 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.535642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:59.535791 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:59.536387 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:59.536397 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:59.537897 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:59.539579 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:59.539587 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:59.539590 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:59.539931 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:59.540156 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:59.540156 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:59.540157 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:59.540322 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:59.540601 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:59.540609 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:59.540806 (+ 197us) client_negotiation.cc:769] Sending connection context
0927 18:41:59.540943 (+ 137us) client_negotiation.cc:240] Negotiation successful
0927 18:41:59.541145 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":147,"threads_started":1}
I20260927 18:41:59.542414 27973 heartbeater.cc:347] Connected to a master server at 127.11.251.254:41973
I20260927 18:41:59.542532 27973 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:59.542773 27973 heartbeater.cc:511] Master 127.11.251.254:41973 requested a full tablet report, sending...
I20260927 18:41:59.543279 27648 ts_manager.cc:194] Registered new tserver with Master: 1d2806f483c04b6592f08e3bb1f7f62d (127.11.251.194:34265)
I20260927 18:41:59.543741 27648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:46475
WARNING: no policy specified for kudu/127.11.251.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
I20260927 18:41:59.564399 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41973
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:59.656625 27978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:59.656903 27978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:59.656944 27978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:59.658268 27978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:59.658331 27978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:59.658381 27978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:41:59.659893 27978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:41:59.660169 27978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:59.660422 27978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:59.665729 27984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:59.665748 27986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:59.665791 27978 server_base.cc:1034] running on GCE node
W20260927 18:41:59.665890 27983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:59.666124 27978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:59.666363 27978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:59.667508 27978 hybrid_clock.cc:648] HybridClock initialized: now 1790534519667489 us; error 40 us; skew 500 ppm
Sep 27 18:41:59 dist-test-slave-z47j krb5kdc[27603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534519, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:59.670347 27978 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:41:59.670871 27978 webserver.cc:460] Webserver started at http://127.11.251.195:35739/ using document root <none> and password file <none>
I20260927 18:41:59.671106 27978 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:59.671154 27978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:59.671298 27978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:59.672438 27978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "5bcf0be3cf514817ad9a3ffe28564c37"
format_stamp: "Formatted at 2026-09-27 18:41:59 on dist-test-slave-z47j"
server_key: "990d41eec035dbae59c99059d96aa9bf"
server_key_iv: "495f54178ea56b821a00e2df25cef776"
server_key_version: "encryptionkey@0"
I20260927 18:41:59.672824 27978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "5bcf0be3cf514817ad9a3ffe28564c37"
format_stamp: "Formatted at 2026-09-27 18:41:59 on dist-test-slave-z47j"
server_key: "990d41eec035dbae59c99059d96aa9bf"
server_key_iv: "495f54178ea56b821a00e2df25cef776"
server_key_version: "encryptionkey@0"
I20260927 18:41:59.674348 27978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:41:59.675222 27993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:59.675452 27978 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:41:59.675547 27978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "5bcf0be3cf514817ad9a3ffe28564c37"
format_stamp: "Formatted at 2026-09-27 18:41:59 on dist-test-slave-z47j"
server_key: "990d41eec035dbae59c99059d96aa9bf"
server_key_iv: "495f54178ea56b821a00e2df25cef776"
server_key_version: "encryptionkey@0"
I20260927 18:41:59.675626 27978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:59.690752 27978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:59.693274 27978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:59.693459 27978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:59.693786 27978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:59.694146 27978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:59.694200 27978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:59.694245 27978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:59.694272 27978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:59.701308 27978 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:38775
I20260927 18:41:59.701354 28106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:38775 every 8 connection(s)
I20260927 18:41:59.701800 27978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
Sep 27 18:41:59 dist-test-slave-z47j krb5kdc[27603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534519, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:41:59.710358 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 27978
I20260927 18:41:59.710474 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:41:59.710763 12271 external_mini_cluster.cc:1428] Setting key b3276bc4ea1ff18473e3ba73f3408395
I20260927 18:41:59.712176 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:59.702610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:42105 (local address 127.11.251.254:41973)
0927 18:41:59.702682 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:59.702685 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:59.702935 (+ 250us) server_negotiation.cc:407] Connection header received
0927 18:41:59.703541 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:59.703543 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:59.703557 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:59.703625 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:59.704676 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.705245 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.706347 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.706554 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.706635 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:59.708926 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:59.708928 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:59.708929 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:59.708936 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:59.710296 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:59.710766 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:59.710768 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:59.710768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:59.710829 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:59.711145 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:59.711147 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:59.711147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:59.711338 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:59.711427 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:59.711960 (+ 533us) server_negotiation.cc:299] Negotiation successful
0927 18:41:59.712056 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 18:41:59.712409 28109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:59.702603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:42105)
0927 18:41:59.702799 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:59.702840 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:59.703427 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:59.703709 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:59.703713 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:59.704347 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:59.704575 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:59.704580 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.705362 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.705363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:59.706249 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:59.706252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.706647 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.706648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:59.706782 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:59.707345 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:59.707353 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:59.708830 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:59.710377 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:59.710383 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:59.710386 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:59.710681 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:59.710916 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:59.710917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:59.710918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:59.711052 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:59.711416 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:59.711423 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:59.711560 (+ 137us) client_negotiation.cc:769] Sending connection context
0927 18:41:59.711924 (+ 364us) client_negotiation.cc:240] Negotiation successful
0927 18:41:59.712129 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
I20260927 18:41:59.712890 28107 heartbeater.cc:347] Connected to a master server at 127.11.251.254:41973
I20260927 18:41:59.712991 28107 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:59.713177 28107 heartbeater.cc:511] Master 127.11.251.254:41973 requested a full tablet report, sending...
I20260927 18:41:59.713601 27648 ts_manager.cc:194] Registered new tserver with Master: 5bcf0be3cf514817ad9a3ffe28564c37 (127.11.251.195:38775)
I20260927 18:41:59.713995 27648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:42105
I20260927 18:41:59.722482 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:41:59.730372 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:59.724015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51280 (local address 127.11.251.254:41973)
0927 18:41:59.724077 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:59.724079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:59.724214 (+ 135us) server_negotiation.cc:407] Connection header received
0927 18:41:59.724295 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:59.724295 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:59.724307 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:59.724365 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:59.725018 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.725528 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.726275 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.726487 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.726568 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:59.727843 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:59.727845 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:59.727846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:59.727853 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:59.729104 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:59.729486 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:59.729488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:59.729488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:59.729546 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:59.729787 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:59.729788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:59.729789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:59.729962 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:59.730077 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:59.730182 (+ 105us) server_negotiation.cc:299] Negotiation successful
0927 18:41:59.730263 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 18:41:59.731947 27648 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:41:59.733964 27648 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:41:59.741051 28119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:59.736382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:34265 (local address 127.0.0.1:56398)
0927 18:41:59.736574 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:59.736619 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:59.736728 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:59.737226 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:59.737228 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:59.737233 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:59.737308 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:59.737310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.738745 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.738746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:59.739826 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:59.739828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.740744 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.740746 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:59.740839 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:59.740843 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:41:59.740903 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 18:41:59.740954 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":6,"thread_start_us":57,"threads_started":1}
I20260927 18:41:59.741073 28121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:59.736527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56398 (local address 127.11.251.194:34265)
0927 18:41:59.737021 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:59.737026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:59.737038 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:41:59.737083 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:59.737085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:59.737103 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:59.737387 (+ 284us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:59.737471 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.738643 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.739906 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.740618 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.740739 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:59.740798 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:59.740925 (+ 127us) server_negotiation.cc:299] Negotiation successful
0927 18:41:59.740975 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":99,"threads_started":1}
I20260927 18:41:59.741183 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:59.736738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37250 (local address 127.11.251.193:46563)
0927 18:41:59.736917 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:59.736922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:59.737141 (+ 219us) server_negotiation.cc:407] Connection header received
0927 18:41:59.737194 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:59.737197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:59.737215 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:59.737295 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:59.737510 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.738282 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.739746 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.740486 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.740569 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:59.740633 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:59.741007 (+ 374us) server_negotiation.cc:299] Negotiation successful
0927 18:41:59.741075 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260927 18:41:59.741430 28118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:59.736420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:46563 (local address 127.0.0.1:37250)
0927 18:41:59.736929 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:59.736940 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:59.737003 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:59.737375 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:59.737376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:59.737378 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:59.737436 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:59.737438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.738386 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.738387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:59.739661 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:59.739664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.740573 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.740575 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:59.740709 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:59.740738 (+ 29us) client_negotiation.cc:769] Sending connection context
0927 18:41:59.740815 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 18:41:59.740905 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
I20260927 18:41:59.742542 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:59.736399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:38775 (local address 127.0.0.1:39842)
0927 18:41:59.736860 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:59.736869 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:59.736913 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:59.737224 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:59.737228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:59.737230 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:59.737308 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:59.737311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.739906 (+ 2595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.739906 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:59.740623 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:59.740624 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.742213 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.742214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:59.742307 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:59.742311 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:41:59.742402 (+ 91us) client_negotiation.cc:240] Negotiation successful
0927 18:41:59.742454 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":117,"threads_started":1}
I20260927 18:41:59.742599 28123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:59.736742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39842 (local address 127.11.251.195:38775)
0927 18:41:59.737025 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:59.737029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:59.737038 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:41:59.737083 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:59.737085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:59.737103 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:59.737387 (+ 284us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:59.737471 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.738632 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.740814 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.742098 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.742206 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:59.742265 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:59.742425 (+ 160us) server_negotiation.cc:299] Negotiation successful
0927 18:41:59.742484 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":102,"threads_started":1}
I20260927 18:41:59.742415 27773 tablet_service.cc:1467] Processing CreateTablet for tablet 6f0bb1aeeadb4b03b2581dfb56c3bc6e (DEFAULT_TABLE table=test-table [id=58cabf562eb94feabd44d25f75711c0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:59.742416 27907 tablet_service.cc:1467] Processing CreateTablet for tablet 6f0bb1aeeadb4b03b2581dfb56c3bc6e (DEFAULT_TABLE table=test-table [id=58cabf562eb94feabd44d25f75711c0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:59.742940 27773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f0bb1aeeadb4b03b2581dfb56c3bc6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:59.742941 27907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f0bb1aeeadb4b03b2581dfb56c3bc6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:59.743254 28041 tablet_service.cc:1467] Processing CreateTablet for tablet 6f0bb1aeeadb4b03b2581dfb56c3bc6e (DEFAULT_TABLE table=test-table [id=58cabf562eb94feabd44d25f75711c0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:59.743530 28041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f0bb1aeeadb4b03b2581dfb56c3bc6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:59.747082 28124 tablet_bootstrap.cc:492] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37: Bootstrap starting.
I20260927 18:41:59.747408 28126 tablet_bootstrap.cc:492] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d: Bootstrap starting.
I20260927 18:41:59.748076 28126 tablet_bootstrap.cc:654] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:59.748076 28124 tablet_bootstrap.cc:654] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:59.748780 28126 log.cc:824] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:59.748781 28124 log.cc:824] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:59.749135 28125 tablet_bootstrap.cc:492] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522: Bootstrap starting.
I20260927 18:41:59.749518 28126 tablet_bootstrap.cc:492] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d: No bootstrap required, opened a new log
I20260927 18:41:59.749657 28126 ts_tablet_manager.cc:1397] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:41:59.749755 28125 tablet_bootstrap.cc:654] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:59.750239 28125 log.cc:824] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:59.750921 28125 tablet_bootstrap.cc:492] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522: No bootstrap required, opened a new log
I20260927 18:41:59.750975 28125 ts_tablet_manager.cc:1397] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:41:59.751502 28126 raft_consensus.cc:348] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:41:59.751675 28126 raft_consensus.cc:374] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:59.751731 28126 raft_consensus.cc:729] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d2806f483c04b6592f08e3bb1f7f62d, State: Initialized, Role: FOLLOWER
I20260927 18:41:59.751858 28126 consensus_queue.cc:260] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:41:59.752200 28126 ts_tablet_manager.cc:1428] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:41:59.752213 27973 heartbeater.cc:503] Master 127.11.251.254:41973 was elected leader, sending a full tablet report...
I20260927 18:41:59.752848 28124 tablet_bootstrap.cc:492] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37: No bootstrap required, opened a new log
I20260927 18:41:59.752929 28124 ts_tablet_manager.cc:1397] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260927 18:41:59.752808 28125 raft_consensus.cc:348] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:41:59.752970 28125 raft_consensus.cc:374] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:59.753014 28125 raft_consensus.cc:729] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5984283f154ce896d6ade588666522, State: Initialized, Role: FOLLOWER
I20260927 18:41:59.753113 28125 consensus_queue.cc:260] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:41:59.753427 28125 ts_tablet_manager.cc:1428] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 18:41:59.753597 27839 heartbeater.cc:503] Master 127.11.251.254:41973 was elected leader, sending a full tablet report...
I20260927 18:41:59.754992 28124 raft_consensus.cc:348] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:41:59.755162 28124 raft_consensus.cc:374] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:59.755219 28124 raft_consensus.cc:729] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bcf0be3cf514817ad9a3ffe28564c37, State: Initialized, Role: FOLLOWER
I20260927 18:41:59.755329 28124 consensus_queue.cc:260] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:41:59.755600 28124 ts_tablet_manager.cc:1428] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:41:59.755630 28107 heartbeater.cc:503] Master 127.11.251.254:41973 was elected leader, sending a full tablet report...
W20260927 18:41:59.780707 27974 tablet.cc:2369] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:41:59.809010 28132 raft_consensus.cc:482] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:41:59.809206 28132 raft_consensus.cc:504] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:41:59.809859 28132 leader_election.cc:288] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d2806f483c04b6592f08e3bb1f7f62d (127.11.251.194:34265), 3e5984283f154ce896d6ade588666522 (127.11.251.193:46563)
I20260927 18:41:59.813198 28109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:59.810126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:46563 (local address 127.11.251.195:57213)
0927 18:41:59.810198 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:59.810214 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:59.810286 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:59.810451 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:59.810452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:59.810453 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:59.810552 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:59.810554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.811370 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.811371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:59.812233 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:59.812235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.812886 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.812887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:59.812978 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:59.812982 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:41:59.813056 (+ 74us) client_negotiation.cc:240] Negotiation successful
0927 18:41:59.813109 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260927 18:41:59.813199 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:59.810260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:57213 (local address 127.11.251.193:46563)
0927 18:41:59.810323 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:59.810325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:59.810337 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:41:59.810369 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:59.810370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:59.810379 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:59.810436 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:59.810677 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.811267 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.812346 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.812798 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.812890 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:59.812903 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:59.813064 (+ 161us) server_negotiation.cc:299] Negotiation successful
0927 18:41:59.813124 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:41:59.813531 28121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:59.810247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:55599 (local address 127.11.251.194:34265)
0927 18:41:59.810327 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:59.810329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:59.810855 (+ 526us) server_negotiation.cc:407] Connection header received
0927 18:41:59.810935 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:59.810937 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:59.810954 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:59.811023 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:59.811213 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.811717 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.812680 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.813114 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.813209 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:59.813218 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:59.813405 (+ 187us) server_negotiation.cc:299] Negotiation successful
0927 18:41:59.813455 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:41:59.813704 27793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f0bb1aeeadb4b03b2581dfb56c3bc6e" candidate_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5984283f154ce896d6ade588666522" is_pre_election: true
I20260927 18:41:59.813833 28133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:59.810127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:34265 (local address 127.11.251.195:55599)
0927 18:41:59.810759 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:59.810775 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:59.810863 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:59.811026 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:59.811027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:59.811045 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:59.811133 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:59.811136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.811810 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.811811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:59.812578 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:59.812580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.813209 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.813210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:59.813292 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:59.813297 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:41:59.813707 (+ 410us) client_negotiation.cc:240] Negotiation successful
0927 18:41:59.813759 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":347,"thread_start_us":205,"threads_started":1}
I20260927 18:41:59.813910 27793 raft_consensus.cc:2454] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bcf0be3cf514817ad9a3ffe28564c37 in term 0.
I20260927 18:41:59.814172 27996 leader_election.cc:302] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5984283f154ce896d6ade588666522, 5bcf0be3cf514817ad9a3ffe28564c37; no voters:
I20260927 18:41:59.814239 27927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f0bb1aeeadb4b03b2581dfb56c3bc6e" candidate_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" is_pre_election: true
I20260927 18:41:59.814303 28132 raft_consensus.cc:2781] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:41:59.814364 28132 raft_consensus.cc:482] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:41:59.814384 28132 raft_consensus.cc:3037] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:59.814430 27927 raft_consensus.cc:2454] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bcf0be3cf514817ad9a3ffe28564c37 in term 0.
I20260927 18:41:59.815622 28132 raft_consensus.cc:504] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:41:59.815785 28132 leader_election.cc:288] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 [CANDIDATE]: Term 1 election: Requested vote from peers 1d2806f483c04b6592f08e3bb1f7f62d (127.11.251.194:34265), 3e5984283f154ce896d6ade588666522 (127.11.251.193:46563)
I20260927 18:41:59.815940 27793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f0bb1aeeadb4b03b2581dfb56c3bc6e" candidate_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5984283f154ce896d6ade588666522"
I20260927 18:41:59.815949 27927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f0bb1aeeadb4b03b2581dfb56c3bc6e" candidate_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d2806f483c04b6592f08e3bb1f7f62d"
I20260927 18:41:59.816023 27793 raft_consensus.cc:3037] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:59.816035 27927 raft_consensus.cc:3037] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:59.816886 27793 raft_consensus.cc:2454] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bcf0be3cf514817ad9a3ffe28564c37 in term 1.
I20260927 18:41:59.816895 27927 raft_consensus.cc:2454] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bcf0be3cf514817ad9a3ffe28564c37 in term 1.
I20260927 18:41:59.817058 27996 leader_election.cc:302] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5984283f154ce896d6ade588666522, 5bcf0be3cf514817ad9a3ffe28564c37; no voters:
I20260927 18:41:59.817160 28132 raft_consensus.cc:2781] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:59.817332 28132 raft_consensus.cc:686] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 [term 1 LEADER]: Becoming Leader. State: Replica: 5bcf0be3cf514817ad9a3ffe28564c37, State: Running, Role: LEADER
I20260927 18:41:59.817452 28132 consensus_queue.cc:237] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:41:59.818650 27647 catalog_manager.cc:5495] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5bcf0be3cf514817ad9a3ffe28564c37 (127.11.251.195). New cstate: current_term: 1 leader_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } health_report { overall_health: UNKNOWN } } }
I20260927 18:41:59.839360 28123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:59.836447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39850 (local address 127.11.251.195:38775)
0927 18:41:59.836528 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:59.836530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:59.836540 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:41:59.836598 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:59.836599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:59.836620 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:59.836722 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:41:59.836903 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.837426 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.838333 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:59.838542 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:59.838607 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:59.838869 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:41:59.839013 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:41:59.839104 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:59.839178 (+ 74us) server_negotiation.cc:299] Negotiation successful
0927 18:41:59.839203 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260927 18:41:59.841012 27840 tablet.cc:2369] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:41:59.841703 27793 raft_consensus.cc:1261] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522 [term 1 FOLLOWER]: Refusing update from remote peer 5bcf0be3cf514817ad9a3ffe28564c37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:59.841754 27927 raft_consensus.cc:1261] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d [term 1 FOLLOWER]: Refusing update from remote peer 5bcf0be3cf514817ad9a3ffe28564c37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:59.842082 28132 consensus_queue.cc:1035] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:59.842267 28134 consensus_queue.cc:1035] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:59.844874 28137 mvcc.cc:205] Tried to move back new op lower bound from 7334029393268424704 to 7334029393173254144. Current Snapshot: MvccSnapshot[applied={T|T < 7334029393268424704}]
I20260927 18:41:59.845626 28139 mvcc.cc:205] Tried to move back new op lower bound from 7334029393268424704 to 7334029393173254144. Current Snapshot: MvccSnapshot[applied={T|T < 7334029393268424704}]
I20260927 18:41:59.845645 28138 mvcc.cc:205] Tried to move back new op lower bound from 7334029393268424704 to 7334029393173254144. Current Snapshot: MvccSnapshot[applied={T|T < 7334029393268424704}]
I20260927 18:41:59.847510 27647 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:41:59.847649 27647 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:41:59.848541 27647 catalog_manager.cc:5782] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393: Sending DeleteTablet for 3 replicas of tablet 6f0bb1aeeadb4b03b2581dfb56c3bc6e
I20260927 18:41:59.848975 28041 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f0bb1aeeadb4b03b2581dfb56c3bc6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:59 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:39842
I20260927 18:41:59.849009 27773 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f0bb1aeeadb4b03b2581dfb56c3bc6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:59 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:37250
I20260927 18:41:59.849009 27907 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f0bb1aeeadb4b03b2581dfb56c3bc6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:59 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:56398
I20260927 18:41:59.849283 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 27710
I20260927 18:41:59.853739 28146 tablet_replica.cc:330] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37: stopping tablet replica
I20260927 18:41:59.853879 28146 raft_consensus.cc:2229] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:41:59.854010 28146 raft_consensus.cc:2258] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:41:59.854095 28148 tablet_replica.cc:330] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d: stopping tablet replica
I20260927 18:41:59.854271 28148 raft_consensus.cc:2229] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:41:59.854446 28148 raft_consensus.cc:2258] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:41:59.854594 28146 ts_tablet_manager.cc:1905] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:41:59.855073 28148 ts_tablet_manager.cc:1905] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:41:59.856573 28146 ts_tablet_manager.cc:1918] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:41:59.856657 28146 log.cc:1196] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/6f0bb1aeeadb4b03b2581dfb56c3bc6e
I20260927 18:41:59.856936 28146 ts_tablet_manager.cc:1939] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 5bcf0be3cf514817ad9a3ffe28564c37: Deleting consensus metadata
W20260927 18:41:59.857017 27634 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.251.193:46563 (error 108)
I20260927 18:41:59.857653 27635 catalog_manager.cc:4841] TS 5bcf0be3cf514817ad9a3ffe28564c37 (127.11.251.195:38775): tablet 6f0bb1aeeadb4b03b2581dfb56c3bc6e (table test-table [id=58cabf562eb94feabd44d25f75711c0c]) successfully deleted
I20260927 18:41:59.857910 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 27844
I20260927 18:41:59.857971 28148 ts_tablet_manager.cc:1918] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:41:59.858639 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:59.858086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:46563 (local address 127.0.0.1:37262)
0927 18:41:59.858557 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:59.858601 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.193:46563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449}
W20260927 18:41:59.858914 27634 catalog_manager.cc:4584] TS 3e5984283f154ce896d6ade588666522 (127.11.251.193:46563): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6f0bb1aeeadb4b03b2581dfb56c3bc6e: Network error: Client connection negotiation failed: client connection to 127.11.251.193:46563: connect: Connection refused (error 111)
I20260927 18:41:59.864276 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:59.864126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:34265 (local address 127.0.0.1:56404)
0927 18:41:59.864194 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:59.864230 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.194:34265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 18:41:59.864387 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 27978
I20260927 18:41:59.869432 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 27616
I20260927 18:42:19.876225 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:41973
--webserver_interface=127.11.251.254
--webserver_port=43491
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:41973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:42:19.961706 28150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:19.961918 28150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:19.961941 28150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:19.963028 28150 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 18:42:19.963071 28150 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 18:42:19.963085 28150 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 18:42:19.963097 28150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:42:19.963110 28150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:19.963122 28150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:19.963135 28150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:42:19.963146 28150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:42:19.964556 28150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:41973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:41973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=43491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:42:19.964744 28150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:19.964917 28150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:19.974689 28157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:19.974689 28159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:19.974769 28156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:19.975198 28150 server_base.cc:1034] running on GCE node
I20260927 18:42:19.975451 28150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:19.975781 28150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:19.976965 28150 hybrid_clock.cc:648] HybridClock initialized: now 1790534539976951 us; error 38 us; skew 500 ppm
Sep 27 18:42:19 dist-test-slave-z47j krb5kdc[27603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534539, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:42:19.979928 28150 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:42:19.980515 28150 webserver.cc:460] Webserver started at http://127.11.251.254:43491/ using document root <none> and password file <none>
I20260927 18:42:19.980785 28150 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:19.980893 28150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:19.982512 28150 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:42:19.983279 28166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:19.983490 28150 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:42:19.983608 28150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "af6f2bd605f1446697dd1ecda55f6393"
format_stamp: "Formatted at 2026-09-27 18:41:59 on dist-test-slave-z47j"
server_key: "804a4af3a7f8c4ccfcbd09a4c41a15c5"
server_key_iv: "26c41aa48398c945e26e2a78f2e94950"
server_key_version: "encryptionkey@0"
I20260927 18:42:19.984041 28150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:20.001307 28150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:20.003381 28150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:20.003545 28150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:20.008314 28150 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:41973
I20260927 18:42:20.008446 28218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:41973 every 8 connection(s)
I20260927 18:42:20.008807 28150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:42:20.010828 28219 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:42:20.011595 28219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393: Bootstrap starting.
I20260927 18:42:20.012598 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 28150
I20260927 18:42:20.012966 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:46563
--local_ip_for_outbound_sockets=127.11.251.193
--tserver_master_addrs=127.11.251.254:41973
--webserver_port=37071
--webserver_interface=127.11.251.193
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:42:20.023216 28219 log.cc:824] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:20.025508 28219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:42:20.025918 28219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393: Bootstrap complete.
I20260927 18:42:20.028541 28219 raft_consensus.cc:348] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6f2bd605f1446697dd1ecda55f6393" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41973 } }
I20260927 18:42:20.028796 28219 raft_consensus.cc:729] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: af6f2bd605f1446697dd1ecda55f6393, State: Initialized, Role: FOLLOWER
I20260927 18:42:20.028993 28219 consensus_queue.cc:260] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6f2bd605f1446697dd1ecda55f6393" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41973 } }
I20260927 18:42:20.029098 28219 raft_consensus.cc:388] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:42:20.029152 28219 raft_consensus.cc:482] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:42:20.029232 28219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 1 FOLLOWER]: Advancing to term 2
I20260927 18:42:20.030505 28219 raft_consensus.cc:504] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6f2bd605f1446697dd1ecda55f6393" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41973 } }
I20260927 18:42:20.030634 28219 leader_election.cc:302] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af6f2bd605f1446697dd1ecda55f6393; no voters:
I20260927 18:42:20.030866 28219 leader_election.cc:288] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 18:42:20.030910 28223 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 18:42:20.031039 28223 raft_consensus.cc:686] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 2 LEADER]: Becoming Leader. State: Replica: af6f2bd605f1446697dd1ecda55f6393, State: Running, Role: LEADER
I20260927 18:42:20.031251 28219 sys_catalog.cc:564] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:42:20.031221 28223 consensus_queue.cc:237] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6f2bd605f1446697dd1ecda55f6393" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41973 } }
I20260927 18:42:20.031813 28224 sys_catalog.cc:455] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "af6f2bd605f1446697dd1ecda55f6393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6f2bd605f1446697dd1ecda55f6393" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41973 } } }
I20260927 18:42:20.031915 28224 sys_catalog.cc:458] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [sys.catalog]: This master's current role is: LEADER
I20260927 18:42:20.032075 28225 sys_catalog.cc:455] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af6f2bd605f1446697dd1ecda55f6393. Latest consensus state: current_term: 2 leader_uuid: "af6f2bd605f1446697dd1ecda55f6393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6f2bd605f1446697dd1ecda55f6393" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41973 } } }
I20260927 18:42:20.032130 28225 sys_catalog.cc:458] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [sys.catalog]: This master's current role is: LEADER
I20260927 18:42:20.032523 28232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:42:20.033566 28232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:42:20.033715 28232 catalog_manager.cc:1259] Loaded cluster ID: e2cee189163347e1b3cf43ccdfa85f11
I20260927 18:42:20.033757 28232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:42:20.035003 28232 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:42:20.035202 28232 catalog_manager.cc:5879] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393: Loaded TSK: 0
I20260927 18:42:20.050002 28232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393: Generated new TSK 1
I20260927 18:42:20.050132 28232 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 18:42:20.101536 28221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:20.101768 28221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:20.101792 28221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:20.102994 28221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:20.103041 28221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:20.103068 28221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:42:20.104514 28221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:46563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=37071
--tserver_master_addrs=127.11.251.254:41973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:42:20.104748 28221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:20.105007 28221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:20.111613 28249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:20.111649 28247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:20.111609 28246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:20.112298 28221 server_base.cc:1034] running on GCE node
I20260927 18:42:20.112498 28221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:20.112741 28221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:20.113916 28221 hybrid_clock.cc:648] HybridClock initialized: now 1790534540113898 us; error 39 us; skew 500 ppm
Sep 27 18:42:20 dist-test-slave-z47j krb5kdc[27603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534540, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:42:20.116693 28221 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:42:20.117166 28221 webserver.cc:460] Webserver started at http://127.11.251.193:37071/ using document root <none> and password file <none>
I20260927 18:42:20.117422 28221 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:20.117484 28221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:20.118952 28221 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:42:20.119681 28256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:20.119869 28221 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:42:20.119983 28221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "3e5984283f154ce896d6ade588666522"
format_stamp: "Formatted at 2026-09-27 18:41:59 on dist-test-slave-z47j"
server_key: "73443c1737cfb89734313f03b7891039"
server_key_iv: "251295dc151a5b5fa3248faca4aa16f3"
server_key_version: "encryptionkey@0"
I20260927 18:42:20.120329 28221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:20.130695 28221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:20.133181 28221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:20.133351 28221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:20.133661 28221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:42:20.134290 28263 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 18:42:20.136650 28221 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 18:42:20.136739 28221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260927 18:42:20.136796 28221 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 18:42:20.137562 28221 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 18:42:20.137670 28221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260927 18:42:20.137801 28263 tablet_bootstrap.cc:492] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522: Bootstrap starting.
I20260927 18:42:20.145318 28221 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:46563
I20260927 18:42:20.145740 28370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:46563 every 8 connection(s)
I20260927 18:42:20.145785 28221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:42:20.149461 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 28221
I20260927 18:42:20.149825 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:34265
--local_ip_for_outbound_sockets=127.11.251.194
--tserver_master_addrs=127.11.251.254:41973
--webserver_port=33107
--webserver_interface=127.11.251.194
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:42:20.151475 28263 log.cc:824] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522: Log is configured to *not* fsync() on all Append() calls
Sep 27 18:42:20 dist-test-slave-z47j krb5kdc[27603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534540, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:42:20.154793 28263 tablet_bootstrap.cc:492] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:42:20.155467 28263 tablet_bootstrap.cc:492] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522: Bootstrap complete.
I20260927 18:42:20.156845 28263 ts_tablet_manager.cc:1397] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.015s
I20260927 18:42:20.158457 28263 raft_consensus.cc:348] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:42:20.158964 28263 raft_consensus.cc:729] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5984283f154ce896d6ade588666522, State: Initialized, Role: FOLLOWER
I20260927 18:42:20.159216 28373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.146586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:37163)
0927 18:42:20.146864 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:20.146908 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:20.147362 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:20.147871 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:20.147874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:20.148168 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:20.148366 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:20.148370 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.149756 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.149758 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:20.151084 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:20.151087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.152072 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.152073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:20.152222 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:20.152803 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:20.152812 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:20.154487 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:20.157030 (+ 2543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:20.157038 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:20.157041 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:20.157389 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:20.157744 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:20.157745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:20.157745 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:20.157887 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:20.158409 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:20.158416 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:20.158609 (+ 193us) client_negotiation.cc:769] Sending connection context
0927 18:42:20.158710 (+ 101us) client_negotiation.cc:240] Negotiation successful
0927 18:42:20.158912 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":99,"threads_started":1}
I20260927 18:42:20.159219 28263 consensus_queue.cc:260] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:42:20.159896 28374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.146694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:37163 (local address 127.11.251.254:41973)
0927 18:42:20.147511 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:20.147518 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:20.147545 (+ 27us) server_negotiation.cc:407] Connection header received
0927 18:42:20.147649 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:20.147653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:20.147692 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:20.147842 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:20.148486 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.149594 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.151207 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.151843 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.151953 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:20.154621 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:20.154626 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:20.154626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:20.154641 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:20.156920 (+ 2279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:20.157497 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:20.157501 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:20.157501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:20.157655 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:20.157974 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:20.157975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:20.157975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:20.158313 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:20.159416 (+ 1103us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:20.159497 (+ 81us) server_negotiation.cc:299] Negotiation successful
0927 18:42:20.159654 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":776,"thread_start_us":72,"threads_started":1}
I20260927 18:42:20.160183 28263 ts_tablet_manager.cc:1428] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:42:20.160861 28371 heartbeater.cc:347] Connected to a master server at 127.11.251.254:41973
I20260927 18:42:20.160980 28371 heartbeater.cc:464] Registering TS with master...
I20260927 18:42:20.161194 28371 heartbeater.cc:511] Master 127.11.251.254:41973 requested a full tablet report, sending...
I20260927 18:42:20.162254 28183 ts_manager.cc:194] Registered new tserver with Master: 3e5984283f154ce896d6ade588666522 (127.11.251.193:46563)
I20260927 18:42:20.163599 28183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:37163
I20260927 18:42:20.164906 28371 heartbeater.cc:503] Master 127.11.251.254:41973 was elected leader, sending a full tablet report...
I20260927 18:42:20.169857 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.163387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33456 (local address 127.11.251.193:46563)
0927 18:42:20.163553 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:20.163559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:20.165340 (+ 1781us) server_negotiation.cc:407] Connection header received
0927 18:42:20.165487 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:20.165490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:20.165507 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:20.165558 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:42:20.165857 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.166921 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.168608 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.169352 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.169439 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:20.169496 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:20.169721 (+ 225us) server_negotiation.cc:299] Negotiation successful
0927 18:42:20.169769 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":104,"threads_started":1}
I20260927 18:42:20.169857 28378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.163042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:46563 (local address 127.0.0.1:33456)
0927 18:42:20.165238 (+ 2196us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:20.165279 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:20.165383 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:20.165681 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:20.165683 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:20.165687 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:42:20.165760 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:20.165762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.167029 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.167030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:20.168497 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:20.168501 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.169463 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.169464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:20.169563 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:20.169621 (+ 58us) client_negotiation.cc:769] Sending connection context
0927 18:42:20.169701 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 18:42:20.169761 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2160,"thread_start_us":138,"threads_started":1}
I20260927 18:42:20.170574 28305 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f0bb1aeeadb4b03b2581dfb56c3bc6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:59 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:33456
I20260927 18:42:20.171120 28380 tablet_replica.cc:330] stopping tablet replica
I20260927 18:42:20.171267 28380 raft_consensus.cc:2229] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:42:20.171360 28380 raft_consensus.cc:2258] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:42:20.171813 28380 ts_tablet_manager.cc:1905] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:42:20.174363 28380 ts_tablet_manager.cc:1918] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:42:20.174441 28380 log.cc:1196] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/wal/wals/6f0bb1aeeadb4b03b2581dfb56c3bc6e
I20260927 18:42:20.174667 28380 ts_tablet_manager.cc:1939] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 3e5984283f154ce896d6ade588666522: Deleting consensus metadata
I20260927 18:42:20.175247 28169 catalog_manager.cc:4841] TS 3e5984283f154ce896d6ade588666522 (127.11.251.193:46563): tablet 6f0bb1aeeadb4b03b2581dfb56c3bc6e (table test-table [id=58cabf562eb94feabd44d25f75711c0c]) successfully deleted
W20260927 18:42:20.241500 28375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:20.241781 28375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:20.241811 28375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:20.242933 28375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:20.242975 28375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:20.243001 28375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:42:20.244388 28375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:34265
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=33107
--tserver_master_addrs=127.11.251.254:41973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:42:20.244580 28375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:20.244755 28375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:20.250303 28385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:20.250286 28386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:20.250672 28375 server_base.cc:1034] running on GCE node
W20260927 18:42:20.250286 28388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:20.251020 28375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:20.251291 28375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:20.252449 28375 hybrid_clock.cc:648] HybridClock initialized: now 1790534540252421 us; error 42 us; skew 500 ppm
Sep 27 18:42:20 dist-test-slave-z47j krb5kdc[27603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534540, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:42:20.255213 28375 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:42:20.255729 28375 webserver.cc:460] Webserver started at http://127.11.251.194:33107/ using document root <none> and password file <none>
I20260927 18:42:20.255993 28375 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:20.256076 28375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:20.257555 28375 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:42:20.258478 28395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:20.258750 28375 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:42:20.258858 28375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "1d2806f483c04b6592f08e3bb1f7f62d"
format_stamp: "Formatted at 2026-09-27 18:41:59 on dist-test-slave-z47j"
server_key: "9e43ea0d5ef162db2b81f9a7786b70c0"
server_key_iv: "0402299b95d605748accd4ebf664b60b"
server_key_version: "encryptionkey@0"
I20260927 18:42:20.259224 28375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:20.280157 28375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:20.282531 28375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:20.282702 28375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:20.282943 28375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:42:20.283440 28402 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260927 18:42:20.285229 28402 ts_tablet_manager.cc:1852] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260927 18:42:20.285583 28402 ts_tablet_manager.cc:1905] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:42:20.288347 28402 ts_tablet_manager.cc:1918] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:42:20.288467 28402 log.cc:1196] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/6f0bb1aeeadb4b03b2581dfb56c3bc6e
I20260927 18:42:20.288795 28402 ts_tablet_manager.cc:1939] T 6f0bb1aeeadb4b03b2581dfb56c3bc6e P 1d2806f483c04b6592f08e3bb1f7f62d: Deleting consensus metadata
I20260927 18:42:20.289042 28375 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260927 18:42:20.289112 28375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260927 18:42:20.289141 28375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:42:20.289157 28375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:20.296424 28375 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:34265
I20260927 18:42:20.296519 28509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:34265 every 8 connection(s)
I20260927 18:42:20.296819 28375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:42:20.298769 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 28375
I20260927 18:42:20.299094 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:38775
--local_ip_for_outbound_sockets=127.11.251.195
--tserver_master_addrs=127.11.251.254:41973
--webserver_port=35739
--webserver_interface=127.11.251.195
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:42:20 dist-test-slave-z47j krb5kdc[27603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534540, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:42:20.308233 28512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.297693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:48937)
0927 18:42:20.297910 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:20.297954 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:20.298653 (+ 699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:20.298956 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:20.298960 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:20.299360 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:20.299598 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:20.299602 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.300496 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.300498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:20.301413 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:20.301417 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.301965 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.301966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:20.302082 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:20.302742 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:20.302751 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:20.304626 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:20.306793 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:20.306798 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:20.306800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:20.307030 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:20.307279 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:20.307280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:20.307280 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:20.307381 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:20.307712 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:20.307716 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:20.307827 (+ 111us) client_negotiation.cc:769] Sending connection context
0927 18:42:20.307916 (+ 89us) client_negotiation.cc:240] Negotiation successful
0927 18:42:20.308039 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":87,"threads_started":1}
I20260927 18:42:20.308248 28374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.297739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:48937 (local address 127.11.251.254:41973)
0927 18:42:20.297849 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:20.297853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:20.298065 (+ 212us) server_negotiation.cc:407] Connection header received
0927 18:42:20.298776 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:20.298778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:20.298798 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:20.298851 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:20.299714 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.300367 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.301510 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.301872 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.301955 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:20.304741 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:20.304743 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:20.304744 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:20.304755 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:20.306691 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:20.307119 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:20.307120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:20.307121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:20.307199 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:20.307462 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:20.307463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:20.307464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:20.307631 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:20.307717 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:20.307970 (+ 253us) server_negotiation.cc:299] Negotiation successful
0927 18:42:20.308085 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 18:42:20.309118 28510 heartbeater.cc:347] Connected to a master server at 127.11.251.254:41973
I20260927 18:42:20.309276 28510 heartbeater.cc:464] Registering TS with master...
I20260927 18:42:20.309551 28510 heartbeater.cc:511] Master 127.11.251.254:41973 requested a full tablet report, sending...
I20260927 18:42:20.310204 28183 ts_manager.cc:194] Registered new tserver with Master: 1d2806f483c04b6592f08e3bb1f7f62d (127.11.251.194:34265)
I20260927 18:42:20.310802 28183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:48937
W20260927 18:42:20.394083 28513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:20.394346 28513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:20.394372 28513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:20.395578 28513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:20.395628 28513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:20.395664 28513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:42:20.397164 28513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:38775
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=35739
--tserver_master_addrs=127.11.251.254:41973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:42:20.397418 28513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:20.397679 28513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:20.403498 28518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:20.403507 28521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:20.403473 28519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:20.403937 28513 server_base.cc:1034] running on GCE node
I20260927 18:42:20.404168 28513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:20.404415 28513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:20.405617 28513 hybrid_clock.cc:648] HybridClock initialized: now 1790534540405555 us; error 36 us; skew 500 ppm
Sep 27 18:42:20 dist-test-slave-z47j krb5kdc[27603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534540, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:42:20.408371 28513 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:42:20.408919 28513 webserver.cc:460] Webserver started at http://127.11.251.195:35739/ using document root <none> and password file <none>
I20260927 18:42:20.409178 28513 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:20.409276 28513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:20.410962 28513 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:42:20.411764 28528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:20.412006 28513 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:42:20.412123 28513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "5bcf0be3cf514817ad9a3ffe28564c37"
format_stamp: "Formatted at 2026-09-27 18:41:59 on dist-test-slave-z47j"
server_key: "990d41eec035dbae59c99059d96aa9bf"
server_key_iv: "495f54178ea56b821a00e2df25cef776"
server_key_version: "encryptionkey@0"
I20260927 18:42:20.412887 28513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:20.434907 28513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:20.437155 28513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:20.437345 28513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:20.437657 28513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:42:20.438026 28513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:42:20.438083 28513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:20.438129 28513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:42:20.438161 28513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:20.445756 28513 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:38775
I20260927 18:42:20.445814 28641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:38775 every 8 connection(s)
I20260927 18:42:20.446239 28513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
Sep 27 18:42:20 dist-test-slave-z47j krb5kdc[27603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534540, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:42:20.455811 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 28513
I20260927 18:42:20.456207 28374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.447142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:55765 (local address 127.11.251.254:41973)
0927 18:42:20.447213 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:20.447215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:20.447466 (+ 251us) server_negotiation.cc:407] Connection header received
0927 18:42:20.447998 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:20.447999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:20.448010 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:20.448094 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:20.449105 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.449744 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.450728 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.450921 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.450975 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:20.453218 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:20.453219 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:20.453219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:20.453227 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:20.454605 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:20.455011 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:20.455012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:20.455013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:20.455064 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:20.455370 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:20.455371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:20.455371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:20.455478 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:20.455552 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:20.455999 (+ 447us) server_negotiation.cc:299] Negotiation successful
0927 18:42:20.456133 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 18:42:20.456859 28644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.447102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:55765)
0927 18:42:20.447322 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:20.447367 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:20.447901 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:20.448171 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:20.448174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:20.448803 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:20.449013 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:20.449018 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.449826 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.449827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:20.450647 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:20.450649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.450998 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.450999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:20.451105 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:20.451665 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:20.451673 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:20.453119 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:20.454688 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:20.454694 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:20.454696 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:20.454936 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:20.455161 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:20.455162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:20.455163 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:20.455290 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:20.455570 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:20.455577 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:20.455746 (+ 169us) client_negotiation.cc:769] Sending connection context
0927 18:42:20.456410 (+ 664us) client_negotiation.cc:240] Negotiation successful
0927 18:42:20.456606 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":140,"threads_started":1}
I20260927 18:42:20.457481 28642 heartbeater.cc:347] Connected to a master server at 127.11.251.254:41973
I20260927 18:42:20.457629 28642 heartbeater.cc:464] Registering TS with master...
I20260927 18:42:20.457836 28642 heartbeater.cc:511] Master 127.11.251.254:41973 requested a full tablet report, sending...
I20260927 18:42:20.458365 28183 ts_manager.cc:194] Registered new tserver with Master: 5bcf0be3cf514817ad9a3ffe28564c37 (127.11.251.195:38775)
I20260927 18:42:20.458868 28183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:55765
I20260927 18:42:20.463502 28374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.456312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32848 (local address 127.11.251.254:41973)
0927 18:42:20.456344 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:20.456347 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:20.456620 (+ 273us) server_negotiation.cc:407] Connection header received
0927 18:42:20.456714 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:20.456715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:20.456728 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:20.456800 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:20.457717 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.458430 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.459475 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.459682 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.459762 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:20.461001 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:20.461003 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:20.461003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:20.461010 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:20.462302 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:20.462660 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:20.462661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:20.462661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:20.462718 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:20.462925 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:20.462926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:20.462927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:20.463063 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:20.463285 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:20.463324 (+ 39us) server_negotiation.cc:299] Negotiation successful
0927 18:42:20.463401 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9}
I20260927 18:42:20.464387 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:42:20.467638 28374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.464813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32852 (local address 127.11.251.254:41973)
0927 18:42:20.464902 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:20.464905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:20.465059 (+ 154us) server_negotiation.cc:407] Connection header received
0927 18:42:20.465110 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:20.465110 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:20.465160 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:20.465219 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:42:20.465416 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.465925 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.466748 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.466925 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.466986 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:20.467257 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:42:20.467319 (+ 62us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0927 18:42:20.467587 (+ 268us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:32852: token expired
Metrics: {"server-negotiator.queue_time_us":52}
W20260927 18:42:20.467694 28112 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.11.251.254:41973)
W20260927 18:42:20.467736 28374 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:32852: token expired
I20260927 18:42:20.474133 28374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.467988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32856 (local address 127.11.251.254:41973)
0927 18:42:20.468057 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:20.468059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:20.468068 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:42:20.468096 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:20.468097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:20.468108 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:20.468156 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:20.468723 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.469241 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.470098 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.470334 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.470405 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:20.471533 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:20.471534 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:20.471534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:20.471540 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:20.472924 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:20.473265 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:20.473266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:20.473266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:20.473322 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:20.473507 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:20.473508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:20.473508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:20.473732 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:20.473819 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:20.473937 (+ 118us) server_negotiation.cc:299] Negotiation successful
0927 18:42:20.474003 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:42:20.475420 28183 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:42:20.477218 28183 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:42:20.479923 28305 tablet_service.cc:1467] Processing CreateTablet for tablet 9236b4a5871c4931ad266f5d08135454 (DEFAULT_TABLE table=test-table [id=5983bda3b56c48548f41df8cfd2eb178]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:42:20.480155 28305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9236b4a5871c4931ad266f5d08135454. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:42:20.481832 28263 tablet_bootstrap.cc:492] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522: Bootstrap starting.
I20260927 18:42:20.482146 28263 tablet_bootstrap.cc:654] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522: Neither blocks nor log segments found. Creating new log.
I20260927 18:42:20.483210 28263 tablet_bootstrap.cc:492] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522: No bootstrap required, opened a new log
I20260927 18:42:20.483263 28263 ts_tablet_manager.cc:1397] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:42:20.483393 28263 raft_consensus.cc:348] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:42:20.483477 28263 raft_consensus.cc:374] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:42:20.483526 28263 raft_consensus.cc:729] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5984283f154ce896d6ade588666522, State: Initialized, Role: FOLLOWER
I20260927 18:42:20.483572 28263 consensus_queue.cc:260] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:42:20.483592 28378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.479613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:34265 (local address 127.0.0.1:53150)
0927 18:42:20.479660 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:20.479671 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:20.479708 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:20.480376 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:20.480377 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:20.480379 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:42:20.480435 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:20.480437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.481348 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.481349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:20.482300 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:20.482303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.483347 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.483349 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:20.483415 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:20.483420 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:42:20.483450 (+ 30us) client_negotiation.cc:240] Negotiation successful
0927 18:42:20.483512 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260927 18:42:20.483664 28263 ts_tablet_manager.cc:1428] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:20.484038 28648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.479789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53150 (local address 127.11.251.194:34265)
0927 18:42:20.480148 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:20.480154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:20.480166 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:42:20.480218 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:20.480221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:20.480239 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:20.480362 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:42:20.480518 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.481255 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.482391 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.483231 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.483780 (+ 549us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:20.483857 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:20.483888 (+ 31us) server_negotiation.cc:299] Negotiation successful
0927 18:42:20.483934 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":179,"threads_started":1}
I20260927 18:42:20.484171 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.479741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:38775 (local address 127.0.0.1:51830)
0927 18:42:20.479987 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:20.480000 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:20.480052 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:20.480454 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:20.480455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:20.480458 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:42:20.480509 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:20.480511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.481853 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.481854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:20.482960 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:20.482963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.483953 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.483954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:20.484019 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:20.484024 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:42:20.484053 (+ 29us) client_negotiation.cc:240] Negotiation successful
0927 18:42:20.484099 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":89,"threads_started":1}
I20260927 18:42:20.484534 28649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.480028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51830 (local address 127.11.251.195:38775)
0927 18:42:20.480259 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:20.480265 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:20.480277 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:42:20.480320 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:20.480321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:20.480339 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:20.480578 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:42:20.480674 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.481729 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.483072 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.483854 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.484296 (+ 442us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:20.484358 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:20.484400 (+ 42us) server_negotiation.cc:299] Negotiation successful
0927 18:42:20.484448 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":105,"threads_started":1}
I20260927 18:42:20.484782 28444 tablet_service.cc:1467] Processing CreateTablet for tablet 9236b4a5871c4931ad266f5d08135454 (DEFAULT_TABLE table=test-table [id=5983bda3b56c48548f41df8cfd2eb178]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:42:20.485145 28444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9236b4a5871c4931ad266f5d08135454. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:42:20.485308 28576 tablet_service.cc:1467] Processing CreateTablet for tablet 9236b4a5871c4931ad266f5d08135454 (DEFAULT_TABLE table=test-table [id=5983bda3b56c48548f41df8cfd2eb178]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:42:20.485725 28576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9236b4a5871c4931ad266f5d08135454. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:42:20.487143 28402 tablet_bootstrap.cc:492] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d: Bootstrap starting.
I20260927 18:42:20.488238 28402 tablet_bootstrap.cc:654] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d: Neither blocks nor log segments found. Creating new log.
I20260927 18:42:20.488600 28402 log.cc:824] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:20.489482 28402 tablet_bootstrap.cc:492] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d: No bootstrap required, opened a new log
I20260927 18:42:20.489552 28402 ts_tablet_manager.cc:1397] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260927 18:42:20.489902 28652 tablet_bootstrap.cc:492] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: Bootstrap starting.
I20260927 18:42:20.490554 28652 tablet_bootstrap.cc:654] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: Neither blocks nor log segments found. Creating new log.
I20260927 18:42:20.491078 28652 log.cc:824] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:20.491262 28402 raft_consensus.cc:348] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:42:20.491415 28402 raft_consensus.cc:374] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:42:20.491438 28402 raft_consensus.cc:729] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d2806f483c04b6592f08e3bb1f7f62d, State: Initialized, Role: FOLLOWER
I20260927 18:42:20.491580 28402 consensus_queue.cc:260] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:42:20.491940 28510 heartbeater.cc:503] Master 127.11.251.254:41973 was elected leader, sending a full tablet report...
I20260927 18:42:20.492000 28652 tablet_bootstrap.cc:492] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: No bootstrap required, opened a new log
I20260927 18:42:20.492074 28652 ts_tablet_manager.cc:1397] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:42:20.492249 28402 ts_tablet_manager.cc:1428] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:42:20.493912 28652 raft_consensus.cc:348] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:42:20.494046 28652 raft_consensus.cc:374] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:42:20.494068 28652 raft_consensus.cc:729] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bcf0be3cf514817ad9a3ffe28564c37, State: Initialized, Role: FOLLOWER
I20260927 18:42:20.494211 28652 consensus_queue.cc:260] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:42:20.494496 28652 ts_tablet_manager.cc:1428] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 18:42:20.494531 28642 heartbeater.cc:503] Master 127.11.251.254:41973 was elected leader, sending a full tablet report...
W20260927 18:42:20.547480 28511 tablet.cc:2369] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:42:20.652171 28655 raft_consensus.cc:482] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:42:20.652344 28655 raft_consensus.cc:504] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
W20260927 18:42:20.652544 28372 tablet.cc:2369] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:42:20.652841 28655 leader_election.cc:288] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d2806f483c04b6592f08e3bb1f7f62d (127.11.251.194:34265), 3e5984283f154ce896d6ade588666522 (127.11.251.193:46563)
I20260927 18:42:20.656059 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.653149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:44495 (local address 127.11.251.193:46563)
0927 18:42:20.653209 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:20.653212 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:20.653227 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:42:20.653334 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:20.653336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:20.653348 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:20.653405 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:42:20.653652 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.654183 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.655246 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.655653 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.655751 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:20.655763 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:20.655933 (+ 170us) server_negotiation.cc:299] Negotiation successful
0927 18:42:20.655974 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:42:20.656107 28648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.653277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:46637 (local address 127.11.251.194:34265)
0927 18:42:20.653327 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:20.653329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:20.653462 (+ 133us) server_negotiation.cc:407] Connection header received
0927 18:42:20.653534 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:20.653535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:20.653545 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:20.653654 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:42:20.653835 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.654326 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.655345 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.655732 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.655819 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:20.655830 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:20.655993 (+ 163us) server_negotiation.cc:299] Negotiation successful
0927 18:42:20.656032 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:42:20.656111 28644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.653078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:46563 (local address 127.11.251.195:44495)
0927 18:42:20.653141 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:20.653155 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:20.653229 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:20.653428 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:20.653430 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:20.653432 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:42:20.653524 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:20.653529 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.654277 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.654279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:20.655136 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:20.655138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.655748 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.655750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:20.655844 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:20.655847 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:42:20.655924 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 18:42:20.655986 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":42}
I20260927 18:42:20.656276 28656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.653163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:34265 (local address 127.11.251.195:46637)
0927 18:42:20.653377 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:20.653387 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:20.653469 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:20.653664 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:20.653665 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:20.653667 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:42:20.653763 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:20.653765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.654418 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.654419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:20.655253 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:20.655255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.655818 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.655819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:20.655905 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:20.655909 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:42:20.655983 (+ 74us) client_negotiation.cc:240] Negotiation successful
0927 18:42:20.656033 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":139,"threads_started":1}
I20260927 18:42:20.656773 28464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9236b4a5871c4931ad266f5d08135454" candidate_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" is_pre_election: true
I20260927 18:42:20.656821 28319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9236b4a5871c4931ad266f5d08135454" candidate_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5984283f154ce896d6ade588666522" is_pre_election: true
I20260927 18:42:20.656922 28464 raft_consensus.cc:2454] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bcf0be3cf514817ad9a3ffe28564c37 in term 0.
I20260927 18:42:20.657053 28319 raft_consensus.cc:2454] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bcf0be3cf514817ad9a3ffe28564c37 in term 0.
I20260927 18:42:20.657222 28529 leader_election.cc:302] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d2806f483c04b6592f08e3bb1f7f62d, 5bcf0be3cf514817ad9a3ffe28564c37; no voters:
I20260927 18:42:20.657392 28655 raft_consensus.cc:2781] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:42:20.657466 28655 raft_consensus.cc:482] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:42:20.657503 28655 raft_consensus.cc:3037] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:42:20.658303 28655 raft_consensus.cc:504] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:42:20.658468 28655 leader_election.cc:288] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [CANDIDATE]: Term 1 election: Requested vote from peers 1d2806f483c04b6592f08e3bb1f7f62d (127.11.251.194:34265), 3e5984283f154ce896d6ade588666522 (127.11.251.193:46563)
I20260927 18:42:20.658650 28319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9236b4a5871c4931ad266f5d08135454" candidate_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5984283f154ce896d6ade588666522"
I20260927 18:42:20.658659 28464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9236b4a5871c4931ad266f5d08135454" candidate_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d2806f483c04b6592f08e3bb1f7f62d"
I20260927 18:42:20.658779 28464 raft_consensus.cc:3037] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:42:20.658787 28319 raft_consensus.cc:3037] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:42:20.659679 28319 raft_consensus.cc:2454] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bcf0be3cf514817ad9a3ffe28564c37 in term 1.
I20260927 18:42:20.659758 28464 raft_consensus.cc:2454] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bcf0be3cf514817ad9a3ffe28564c37 in term 1.
I20260927 18:42:20.659915 28531 leader_election.cc:302] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5984283f154ce896d6ade588666522, 5bcf0be3cf514817ad9a3ffe28564c37; no voters:
I20260927 18:42:20.660056 28655 raft_consensus.cc:2781] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:42:20.660264 28655 raft_consensus.cc:686] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [term 1 LEADER]: Becoming Leader. State: Replica: 5bcf0be3cf514817ad9a3ffe28564c37, State: Running, Role: LEADER
I20260927 18:42:20.660386 28655 consensus_queue.cc:237] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:42:20.661557 28183 catalog_manager.cc:5495] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5bcf0be3cf514817ad9a3ffe28564c37 (127.11.251.195). New cstate: current_term: 1 leader_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } health_report { overall_health: UNKNOWN } } }
I20260927 18:42:20.671216 28649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.668419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51838 (local address 127.11.251.195:38775)
0927 18:42:20.668482 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:20.668484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:20.668495 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:42:20.668538 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:20.668538 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:20.668555 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:20.668628 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:42:20.668810 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.669256 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.670117 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:20.670339 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:20.670419 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:20.670627 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:42:20.670725 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:42:20.670797 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:20.670937 (+ 140us) server_negotiation.cc:299] Negotiation successful
0927 18:42:20.671078 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:42:20.673213 28464 raft_consensus.cc:1261] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d [term 1 FOLLOWER]: Refusing update from remote peer 5bcf0be3cf514817ad9a3ffe28564c37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:42:20.673240 28319 raft_consensus.cc:1261] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [term 1 FOLLOWER]: Refusing update from remote peer 5bcf0be3cf514817ad9a3ffe28564c37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:42:20.673671 28657 consensus_queue.cc:1035] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:42:20.673870 28655 consensus_queue.cc:1035] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:42:20.677522 28661 mvcc.cc:205] Tried to move back new op lower bound from 7334029478594342912 to 7334029478545928192. Current Snapshot: MvccSnapshot[applied={T|T < 7334029478594342912}]
I20260927 18:42:20.679579 28183 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:42:20.679668 28183 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:42:20.680490 28183 catalog_manager.cc:5782] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393: Sending DeleteTablet for 3 replicas of tablet 9236b4a5871c4931ad266f5d08135454
I20260927 18:42:20.680799 28305 tablet_service.cc:1514] Processing DeleteTablet for tablet 9236b4a5871c4931ad266f5d08135454 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:42:20 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:33456
I20260927 18:42:20.680828 28444 tablet_service.cc:1514] Processing DeleteTablet for tablet 9236b4a5871c4931ad266f5d08135454 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:42:20 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:53150
I20260927 18:42:20.681020 28669 tablet_replica.cc:330] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522: stopping tablet replica
I20260927 18:42:20.681078 28669 raft_consensus.cc:2229] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:42:20.681134 28669 raft_consensus.cc:2258] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:42:20.681164 28576 tablet_service.cc:1514] Processing DeleteTablet for tablet 9236b4a5871c4931ad266f5d08135454 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:42:20 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:51830
I20260927 18:42:20.681409 28671 tablet_replica.cc:330] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: stopping tablet replica
I20260927 18:42:20.681429 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 28221
I20260927 18:42:20.681501 28669 ts_tablet_manager.cc:1905] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:42:20.682219 28671 raft_consensus.cc:2229] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:42:20.682379 28671 raft_consensus.cc:2258] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:42:20.682940 28671 ts_tablet_manager.cc:1905] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:42:20.685055 28671 ts_tablet_manager.cc:1918] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:42:20.685182 28671 log.cc:1196] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/9236b4a5871c4931ad266f5d08135454
I20260927 18:42:20.685539 28671 ts_tablet_manager.cc:1939] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: Deleting consensus metadata
I20260927 18:42:20.686168 28170 catalog_manager.cc:4841] TS 5bcf0be3cf514817ad9a3ffe28564c37 (127.11.251.195:38775): tablet 9236b4a5871c4931ad266f5d08135454 (table test-table [id=5983bda3b56c48548f41df8cfd2eb178]) successfully deleted
W20260927 18:42:20.688235 28169 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.251.193:46563 (error 108)
I20260927 18:42:20.688331 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 28375
I20260927 18:42:20.689016 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.688658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:46563 (local address 127.0.0.1:33468)
0927 18:42:20.688939 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:20.688976 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.193:46563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260}
W20260927 18:42:20.689481 28169 catalog_manager.cc:4584] TS 3e5984283f154ce896d6ade588666522 (127.11.251.193:46563): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9236b4a5871c4931ad266f5d08135454: Network error: Client connection negotiation failed: client connection to 127.11.251.193:46563: connect: Connection refused (error 111)
I20260927 18:42:20.696820 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:20.696703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:34265 (local address 127.0.0.1:53162)
0927 18:42:20.696759 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:20.696793 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.194:34265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 18:42:20.696950 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 28513
I20260927 18:42:20.703136 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 28150
I20260927 18:42:40.710144 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:41973
--webserver_interface=127.11.251.254
--webserver_port=43491
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:41973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:42:40.796252 28673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:40.796478 28673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:40.796501 28673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:40.797708 28673 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 18:42:40.797761 28673 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 18:42:40.797775 28673 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 18:42:40.797787 28673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:42:40.797830 28673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:40.797869 28673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:40.797886 28673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:42:40.797919 28673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:42:40.799458 28673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:41973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:41973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=43491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:42:40.799715 28673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:40.799988 28673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:40.806107 28678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:40.806149 28673 server_base.cc:1034] running on GCE node
W20260927 18:42:40.806126 28681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:40.806130 28679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:40.806629 28673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:40.807000 28673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:40.808198 28673 hybrid_clock.cc:648] HybridClock initialized: now 1790534560808177 us; error 44 us; skew 500 ppm
Sep 27 18:42:40 dist-test-slave-z47j krb5kdc[27603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534560, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:42:40.810946 28673 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:42:40.811483 28673 webserver.cc:460] Webserver started at http://127.11.251.254:43491/ using document root <none> and password file <none>
I20260927 18:42:40.811712 28673 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:40.811781 28673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:40.813227 28673 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:42:40.814078 28688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:40.814324 28673 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 18:42:40.814435 28673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "af6f2bd605f1446697dd1ecda55f6393"
format_stamp: "Formatted at 2026-09-27 18:41:59 on dist-test-slave-z47j"
server_key: "804a4af3a7f8c4ccfcbd09a4c41a15c5"
server_key_iv: "26c41aa48398c945e26e2a78f2e94950"
server_key_version: "encryptionkey@0"
I20260927 18:42:40.814880 28673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:40.830691 28673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:40.833011 28673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:40.833155 28673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:40.838860 28673 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:41973
I20260927 18:42:40.838958 28740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:41973 every 8 connection(s)
I20260927 18:42:40.839411 28673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:42:40.841037 28741 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:42:40.841843 28741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393: Bootstrap starting.
I20260927 18:42:40.845709 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 28673
I20260927 18:42:40.846063 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:46563
--local_ip_for_outbound_sockets=127.11.251.193
--tserver_master_addrs=127.11.251.254:41973
--webserver_port=37071
--webserver_interface=127.11.251.193
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:42:40.851065 28741 log.cc:824] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:40.853400 28741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:42:40.853809 28741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393: Bootstrap complete.
I20260927 18:42:40.856108 28741 raft_consensus.cc:348] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6f2bd605f1446697dd1ecda55f6393" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41973 } }
I20260927 18:42:40.856359 28741 raft_consensus.cc:729] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: af6f2bd605f1446697dd1ecda55f6393, State: Initialized, Role: FOLLOWER
I20260927 18:42:40.856565 28741 consensus_queue.cc:260] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6f2bd605f1446697dd1ecda55f6393" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41973 } }
I20260927 18:42:40.856662 28741 raft_consensus.cc:388] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:42:40.856717 28741 raft_consensus.cc:482] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:42:40.856770 28741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 2 FOLLOWER]: Advancing to term 3
I20260927 18:42:40.857815 28741 raft_consensus.cc:504] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6f2bd605f1446697dd1ecda55f6393" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41973 } }
I20260927 18:42:40.857931 28741 leader_election.cc:302] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af6f2bd605f1446697dd1ecda55f6393; no voters:
I20260927 18:42:40.858137 28741 leader_election.cc:288] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260927 18:42:40.858176 28745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 3 FOLLOWER]: Leader election won for term 3
I20260927 18:42:40.858314 28745 raft_consensus.cc:686] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [term 3 LEADER]: Becoming Leader. State: Replica: af6f2bd605f1446697dd1ecda55f6393, State: Running, Role: LEADER
I20260927 18:42:40.858409 28741 sys_catalog.cc:564] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:42:40.858409 28745 consensus_queue.cc:237] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6f2bd605f1446697dd1ecda55f6393" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41973 } }
I20260927 18:42:40.858749 28745 sys_catalog.cc:455] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af6f2bd605f1446697dd1ecda55f6393. Latest consensus state: current_term: 3 leader_uuid: "af6f2bd605f1446697dd1ecda55f6393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6f2bd605f1446697dd1ecda55f6393" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41973 } } }
I20260927 18:42:40.858821 28745 sys_catalog.cc:458] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [sys.catalog]: This master's current role is: LEADER
I20260927 18:42:40.859059 28746 sys_catalog.cc:455] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "af6f2bd605f1446697dd1ecda55f6393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af6f2bd605f1446697dd1ecda55f6393" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41973 } } }
I20260927 18:42:40.859140 28746 sys_catalog.cc:458] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393 [sys.catalog]: This master's current role is: LEADER
I20260927 18:42:40.859370 28751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:42:40.860532 28751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:42:40.860749 28751 catalog_manager.cc:1259] Loaded cluster ID: e2cee189163347e1b3cf43ccdfa85f11
I20260927 18:42:40.860925 28751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:42:40.862259 28751 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:42:40.862450 28751 catalog_manager.cc:5879] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393: Loaded TSK: 0
I20260927 18:42:40.862498 28751 catalog_manager.cc:5879] T 00000000000000000000000000000000 P af6f2bd605f1446697dd1ecda55f6393: Loaded TSK: 1
F20260927 18:42:40.862746 28751 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f905574c9ad google::LogMessage::Fail() at ??:0
@ 0x7f905574f88a google::LogMessage::SendToLog() at ??:0
@ 0x7f905574c4de google::LogMessage::Flush() at ??:0
@ 0x7f905574da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9057eb9104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f9057eb6d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f905d6a61cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f905d6a648e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f905d6a6594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f905d66cc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f905d676ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f905761a0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f905760f954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f90584736db start_thread at ??:0
@ 0x7f905424a61f clone at ??:0
W20260927 18:42:40.962525 28743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:40.962792 28743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:40.962822 28743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:40.964200 28743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:40.964249 28743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:40.964286 28743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:42:40.965924 28743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:46563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=37071
--tserver_master_addrs=127.11.251.254:41973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:42:40.966228 28743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:40.966522 28743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:40.972368 28783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:40.972289 28782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:40.972275 28785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:40.972906 28743 server_base.cc:1034] running on GCE node
I20260927 18:42:40.973192 28743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:40.973475 28743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:40.974658 28743 hybrid_clock.cc:648] HybridClock initialized: now 1790534560974640 us; error 43 us; skew 500 ppm
Sep 27 18:42:40 dist-test-slave-z47j krb5kdc[27603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534560, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:42:40.977660 28743 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:42:40.978243 28743 webserver.cc:460] Webserver started at http://127.11.251.193:37071/ using document root <none> and password file <none>
I20260927 18:42:40.978510 28743 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:40.978585 28743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:40.980265 28743 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:42:40.981272 28792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:40.981699 28743 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:42:40.981817 28743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "3e5984283f154ce896d6ade588666522"
format_stamp: "Formatted at 2026-09-27 18:41:59 on dist-test-slave-z47j"
server_key: "73443c1737cfb89734313f03b7891039"
server_key_iv: "251295dc151a5b5fa3248faca4aa16f3"
server_key_version: "encryptionkey@0"
I20260927 18:42:40.982304 28743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:41.006304 28743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:41.009130 28743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:41.009303 28743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:41.009676 28743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:42:41.010727 28799 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 18:42:41.013106 28743 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 18:42:41.013211 28743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260927 18:42:41.013273 28743 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 18:42:41.014238 28743 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 18:42:41.014303 28743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260927 18:42:41.014410 28799 tablet_bootstrap.cc:492] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522: Bootstrap starting.
I20260927 18:42:41.016439 28799 log.cc:824] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:41.020071 28799 tablet_bootstrap.cc:492] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:42:41.020439 28799 tablet_bootstrap.cc:492] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522: Bootstrap complete.
I20260927 18:42:41.020591 28799 ts_tablet_manager.cc:1397] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260927 18:42:41.023371 28799 raft_consensus.cc:348] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:42:41.023789 28799 raft_consensus.cc:729] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5984283f154ce896d6ade588666522, State: Initialized, Role: FOLLOWER
I20260927 18:42:41.023993 28799 consensus_queue.cc:260] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:42:41.024318 28799 ts_tablet_manager.cc:1428] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
I20260927 18:42:41.024832 28743 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:46563
I20260927 18:42:41.025081 28908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:46563 every 8 connection(s)
I20260927 18:42:41.025440 28743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:42:41.033442 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 28743
I20260927 18:42:41.033883 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:34265
--local_ip_for_outbound_sockets=127.11.251.194
--tserver_master_addrs=127.11.251.254:41973
--webserver_port=33107
--webserver_interface=127.11.251.194
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:42:41.066525 28911 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:41.026320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:42265)
0927 18:42:41.026531 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:41.026581 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:41.026787 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:41.066252 (+ 39465us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":80,"threads_started":1}
W20260927 18:42:41.066875 28909 heartbeater.cc:644] Failed to heartbeat to 127.11.251.254:41973 (0 consecutive failures): Network error: Failed to ping master at 127.11.251.254:41973: Client connection negotiation failed: client connection to 127.11.251.254:41973: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 18:42:41.067258 28911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:41.067136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:46277)
0927 18:42:41.067193 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:41.067234 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260927 18:42:41.067770 28911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:41.067642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:52255)
0927 18:42:41.067709 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:41.067745 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260927 18:42:41.067970 28909 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 18:42:41.132726 28912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:41.133003 28912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:41.133042 28912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:41.134413 28912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:41.134481 28912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:41.134527 28912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:42:41.136025 28912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:34265
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=33107
--tserver_master_addrs=127.11.251.254:41973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:42:41.136314 28912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:41.136590 28912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:41.142350 28917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:41.142367 28918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:41.142350 28920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:41.142748 28912 server_base.cc:1034] running on GCE node
I20260927 18:42:41.142992 28912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:41.143260 28912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:41.144449 28912 hybrid_clock.cc:648] HybridClock initialized: now 1790534561144432 us; error 49 us; skew 500 ppm
Sep 27 18:42:41 dist-test-slave-z47j krb5kdc[27603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534561, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:42:41.147464 28912 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:42:41.147920 28912 webserver.cc:460] Webserver started at http://127.11.251.194:33107/ using document root <none> and password file <none>
I20260927 18:42:41.148130 28912 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:41.148175 28912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:41.149727 28912 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:42:41.150496 28927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:41.150748 28912 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:42:41.150830 28912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "1d2806f483c04b6592f08e3bb1f7f62d"
format_stamp: "Formatted at 2026-09-27 18:41:59 on dist-test-slave-z47j"
server_key: "9e43ea0d5ef162db2b81f9a7786b70c0"
server_key_iv: "0402299b95d605748accd4ebf664b60b"
server_key_version: "encryptionkey@0"
I20260927 18:42:41.151295 28912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:41.167431 28912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:41.170130 28912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:41.170292 28912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:41.170593 28912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:42:41.171196 28934 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 18:42:41.172816 28912 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 18:42:41.172910 28912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260927 18:42:41.172956 28912 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 18:42:41.173959 28912 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 18:42:41.174000 28912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260927 18:42:41.174147 28934 tablet_bootstrap.cc:492] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d: Bootstrap starting.
I20260927 18:42:41.182363 28912 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:34265
I20260927 18:42:41.182575 29041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:34265 every 8 connection(s)
I20260927 18:42:41.182868 28912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:42:41.184146 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:41.183657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:45709)
0927 18:42:41.183903 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:41.183995 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":142,"threads_started":1}
I20260927 18:42:41.185817 28934 log.cc:824] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d: Log is configured to *not* fsync() on all Append() calls
W20260927 18:42:41.185932 29042 heartbeater.cc:644] Failed to heartbeat to 127.11.251.254:41973 (0 consecutive failures): Network error: Failed to ping master at 127.11.251.254:41973: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
I20260927 18:42:41.186270 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:41.186148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:35747)
0927 18:42:41.186216 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:41.186249 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260927 18:42:41.186738 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:41.186599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:55535)
0927 18:42:41.186669 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:41.186708 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260927 18:42:41.186887 29042 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 18:42:41.188553 28934 tablet_bootstrap.cc:492] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:42:41.188877 28934 tablet_bootstrap.cc:492] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d: Bootstrap complete.
I20260927 18:42:41.189968 28934 ts_tablet_manager.cc:1397] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.008s
I20260927 18:42:41.191290 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 28912
I20260927 18:42:41.191642 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:38775
--local_ip_for_outbound_sockets=127.11.251.195
--tserver_master_addrs=127.11.251.254:41973
--webserver_port=35739
--webserver_interface=127.11.251.195
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:42:41.191588 28934 raft_consensus.cc:348] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:42:41.192036 28934 raft_consensus.cc:729] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d2806f483c04b6592f08e3bb1f7f62d, State: Initialized, Role: FOLLOWER
I20260927 18:42:41.192206 28934 consensus_queue.cc:260] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:42:41.192533 28934 ts_tablet_manager.cc:1428] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260927 18:42:41.192800 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:41.192693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:51201)
0927 18:42:41.192740 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:41.192772 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260927 18:42:41.193090 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:41.193008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:56801)
0927 18:42:41.193050 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:41.193071 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260927 18:42:41.277261 29047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:41.277477 29047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:41.277498 29047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:41.278703 29047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:41.278756 29047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:41.278784 29047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:42:41.280232 29047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:38775
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=35739
--tserver_master_addrs=127.11.251.254:41973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:42:41.280484 29047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:41.280740 29047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:41.286701 29052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:41.286734 29053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:41.286701 29055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:41.287165 29047 server_base.cc:1034] running on GCE node
I20260927 18:42:41.287498 29047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:41.287814 29047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:41.289027 29047 hybrid_clock.cc:648] HybridClock initialized: now 1790534561289016 us; error 47 us; skew 500 ppm
Sep 27 18:42:41 dist-test-slave-z47j krb5kdc[27603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534561, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:42:41.292192 29047 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:42:41.292733 29047 webserver.cc:460] Webserver started at http://127.11.251.195:35739/ using document root <none> and password file <none>
I20260927 18:42:41.292971 29047 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:41.293020 29047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:41.294685 29047 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:42:41.295365 29062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:41.295681 29047 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:42:41.295812 29047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "5bcf0be3cf514817ad9a3ffe28564c37"
format_stamp: "Formatted at 2026-09-27 18:41:59 on dist-test-slave-z47j"
server_key: "990d41eec035dbae59c99059d96aa9bf"
server_key_iv: "495f54178ea56b821a00e2df25cef776"
server_key_version: "encryptionkey@0"
I20260927 18:42:41.296248 29047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:41.313634 29047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:41.316052 29047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:41.316239 29047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:41.316537 29047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:42:41.316922 29047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:42:41.316980 29047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:41.317028 29047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:42:41.317056 29047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:41.325124 29047 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:38775
I20260927 18:42:41.325204 29175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:38775 every 8 connection(s)
I20260927 18:42:41.325680 29047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:42:41.327263 29178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:41.326525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:43529)
0927 18:42:41.326758 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:41.326858 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":107,"threads_started":1}
W20260927 18:42:41.327575 29176 heartbeater.cc:644] Failed to heartbeat to 127.11.251.254:41973 (0 consecutive failures): Network error: Failed to ping master at 127.11.251.254:41973: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
I20260927 18:42:41.327975 29178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:41.327826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:34545)
0927 18:42:41.327899 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:41.327941 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260927 18:42:41.328115 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 29047
I20260927 18:42:41.328620 29178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:41.328505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:59889)
0927 18:42:41.328560 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:41.328586 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260927 18:42:41.328778 29176 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 18:42:42.068725 29180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:42.068412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:36553)
0927 18:42:42.068585 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:42.068647 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":87,"threads_started":1}
I20260927 18:42:42.193857 29181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:42.193566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:57695)
0927 18:42:42.193758 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:42.193813 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":77,"threads_started":1}
I20260927 18:42:42.329485 29182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:42.329177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:47317)
0927 18:42:42.329359 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:42.329409 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":75,"threads_started":1}
I20260927 18:42:42.686565 29183 raft_consensus.cc:482] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:42:42.686753 29183 raft_consensus.cc:504] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:42:42.687135 29183 leader_election.cc:288] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d2806f483c04b6592f08e3bb1f7f62d (127.11.251.194:34265), 5bcf0be3cf514817ad9a3ffe28564c37 (127.11.251.195:38775)
Sep 27 18:42:42 dist-test-slave-z47j krb5kdc[27603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534560, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.194@KRBTEST.COM
Sep 27 18:42:42 dist-test-slave-z47j krb5kdc[27603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534560, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.195@KRBTEST.COM
I20260927 18:42:42.708151 29187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:42.687610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:42965 (local address 127.11.251.195:38775)
0927 18:42:42.687754 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:42.687761 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:42.687789 (+ 28us) server_negotiation.cc:407] Connection header received
0927 18:42:42.688391 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:42.688393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:42.688428 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:42.688528 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:42.689381 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:42.690757 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:42.692500 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:42.693138 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:42.693259 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:42.703639 (+ 10380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:42.703643 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:42.703643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:42.703655 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:42.705782 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:42.706506 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:42.706510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:42.706510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:42.706632 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:42.707026 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:42.707027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:42.707027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:42.707361 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:42.707459 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:42.707880 (+ 421us) server_negotiation.cc:299] Negotiation successful
0927 18:42:42.708028 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":52,"threads_started":1}
I20260927 18:42:42.708154 29186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:42.687502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:51807 (local address 127.11.251.194:34265)
0927 18:42:42.687737 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:42.687752 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:42.687783 (+ 31us) server_negotiation.cc:407] Connection header received
0927 18:42:42.687947 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:42.687950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:42.688004 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:42.688154 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:42.689249 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:42.690757 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:42.692500 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:42.693139 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:42.693259 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:42.703365 (+ 10106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:42.703369 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:42.703369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:42.703381 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:42.705794 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:42.706506 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:42.706510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:42.706510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:42.706631 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:42.707026 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:42.707027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:42.707027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:42.707361 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:42.707459 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:42.707880 (+ 421us) server_negotiation.cc:299] Negotiation successful
0927 18:42:42.708028 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":89,"threads_started":1}
I20260927 18:42:42.708225 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:42.687369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:34265 (local address 127.11.251.193:51807)
0927 18:42:42.687497 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:42.687511 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:42.687573 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:42.688240 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:42.688243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:42.688696 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:42.689101 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:42.689111 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:42.690915 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:42.690917 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:42.692390 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:42.692393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:42.693267 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:42.693268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:42.693488 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:42.701002 (+ 7514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:42.701020 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:42.703195 (+ 2175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:42.705901 (+ 2706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:42.705911 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:42.705914 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:42.706395 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:42.706727 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:42.706728 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:42.706728 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:42.706938 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:42.707451 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:42.707459 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:42.707658 (+ 199us) client_negotiation.cc:769] Sending connection context
0927 18:42:42.707797 (+ 139us) client_negotiation.cc:240] Negotiation successful
0927 18:42:42.708061 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":136,"thread_start_us":55,"threads_started":1}
I20260927 18:42:42.708235 29185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:42.687371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:38775 (local address 127.11.251.193:42965)
0927 18:42:42.687558 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:42.687580 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:42.687680 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:42.688548 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:42.688549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:42.688806 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:42.689274 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:42.689276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:42.690919 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:42.690919 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:42.692390 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:42.692393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:42.693266 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:42.693267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:42.693488 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:42.701025 (+ 7537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:42.701035 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:42.703523 (+ 2488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:42.705901 (+ 2378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:42.705911 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:42.705914 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:42.706395 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:42.706727 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:42.706728 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:42.706728 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:42.706938 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:42.707451 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:42.707459 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:42.707658 (+ 199us) client_negotiation.cc:769] Sending connection context
0927 18:42:42.707797 (+ 139us) client_negotiation.cc:240] Negotiation successful
0927 18:42:42.708120 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":191,"thread_start_us":66,"threads_started":1}
I20260927 18:42:42.708873 28996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9236b4a5871c4931ad266f5d08135454" candidate_uuid: "3e5984283f154ce896d6ade588666522" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" is_pre_election: true
I20260927 18:42:42.709085 28996 raft_consensus.cc:2454] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e5984283f154ce896d6ade588666522 in term 1.
I20260927 18:42:42.709462 28793 leader_election.cc:302] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d2806f483c04b6592f08e3bb1f7f62d, 3e5984283f154ce896d6ade588666522; no voters:
I20260927 18:42:42.709647 29183 raft_consensus.cc:2781] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260927 18:42:42.709736 29183 raft_consensus.cc:482] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:42:42.709761 29183 raft_consensus.cc:3037] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [term 1 FOLLOWER]: Advancing to term 2
W20260927 18:42:42.709853 28796 leader_election.cc:341] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5bcf0be3cf514817ad9a3ffe28564c37 (127.11.251.195:38775): Not found: Tablet not found: 9236b4a5871c4931ad266f5d08135454
I20260927 18:42:42.711072 29183 raft_consensus.cc:504] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:42:42.711277 29183 leader_election.cc:288] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [CANDIDATE]: Term 2 election: Requested vote from peers 1d2806f483c04b6592f08e3bb1f7f62d (127.11.251.194:34265), 5bcf0be3cf514817ad9a3ffe28564c37 (127.11.251.195:38775)
I20260927 18:42:42.711613 28996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9236b4a5871c4931ad266f5d08135454" candidate_uuid: "3e5984283f154ce896d6ade588666522" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1d2806f483c04b6592f08e3bb1f7f62d"
I20260927 18:42:42.711725 28996 raft_consensus.cc:3037] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d [term 1 FOLLOWER]: Advancing to term 2
W20260927 18:42:42.711778 28796 leader_election.cc:341] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 5bcf0be3cf514817ad9a3ffe28564c37 (127.11.251.195:38775): Not found: Tablet not found: 9236b4a5871c4931ad266f5d08135454
I20260927 18:42:42.712779 28996 raft_consensus.cc:2454] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5984283f154ce896d6ade588666522 in term 2.
I20260927 18:42:42.712986 28793 leader_election.cc:302] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1d2806f483c04b6592f08e3bb1f7f62d, 3e5984283f154ce896d6ade588666522; no voters: 5bcf0be3cf514817ad9a3ffe28564c37
I20260927 18:42:42.713147 29183 raft_consensus.cc:2781] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 18:42:42.713353 29183 raft_consensus.cc:686] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [term 2 LEADER]: Becoming Leader. State: Replica: 3e5984283f154ce896d6ade588666522, State: Running, Role: LEADER
I20260927 18:42:42.713472 29183 consensus_queue.cc:237] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:42:42.713783 29185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:42.713663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:46711)
0927 18:42:42.713715 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:42.713743 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260927 18:42:43.091132 28996 raft_consensus.cc:1261] T 9236b4a5871c4931ad266f5d08135454 P 1d2806f483c04b6592f08e3bb1f7f62d [term 2 FOLLOWER]: Refusing update from remote peer 3e5984283f154ce896d6ade588666522: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260927 18:42:43.091660 29183 consensus_queue.cc:1035] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260927 18:42:43.092895 29192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:43.092636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:40177)
0927 18:42:43.092761 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:43.092823 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
I20260927 18:42:43.093878 29185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:43.093754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:40793)
0927 18:42:43.093805 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:43.093834 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260927 18:42:43.093952 28796 consensus_peers.cc:487] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522 -> Peer 5bcf0be3cf514817ad9a3ffe28564c37 (127.11.251.195:38775): Couldn't send request to peer 5bcf0be3cf514817ad9a3ffe28564c37. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9236b4a5871c4931ad266f5d08135454. This is attempt 1: this message will repeat every 5th retry.
I20260927 18:42:43.094054 29192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:43.093922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:54815)
0927 18:42:43.093978 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:43.094004 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260927 18:42:43.330271 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:43.330032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:49485)
0927 18:42:43.330185 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:43.330232 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260927 18:42:43.667584 29197 ts_tablet_manager.cc:927] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: Initiating tablet copy from peer 3e5984283f154ce896d6ade588666522 (127.11.251.193:46563)
I20260927 18:42:43.668004 29197 tablet_copy_client.cc:316] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: tablet copy: Beginning tablet copy session from remote peer at address 127.11.251.193:46563
Sep 27 18:42:43 dist-test-slave-z47j krb5kdc[27603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534561, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.193@KRBTEST.COM
I20260927 18:42:43.676505 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:43.668340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:46563 (local address 127.11.251.195:55219)
0927 18:42:43.668422 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:43.668443 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:43.668540 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:43.668845 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:43.668851 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:43.669185 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:43.669254 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:43.669256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:43.670233 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:43.670234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:43.670816 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:43.670818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:43.671331 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:43.671332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:43.671419 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:43.672003 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:43.672008 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:43.673568 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:43.675318 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:43.675326 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:43.675329 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:43.675555 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:43.675766 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:43.675767 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:43.675767 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:43.675823 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:43.676146 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:43.676152 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:43.676214 (+ 62us) client_negotiation.cc:769] Sending connection context
0927 18:42:43.676299 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 18:42:43.676412 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260927 18:42:43.676607 29199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:43.668455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:55219 (local address 127.11.251.193:46563)
0927 18:42:43.668687 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:43.668692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:43.668702 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:42:43.668736 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:43.668738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:43.668752 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:43.668842 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:43.669341 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:43.670142 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:43.670917 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:43.671247 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:43.671334 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:43.673765 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:43.673772 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:43.673773 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:43.673779 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:43.675223 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:43.675635 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:43.675638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:43.675638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:43.675700 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:43.675909 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:43.675910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:43.675910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:43.676063 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:43.676146 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:43.676393 (+ 247us) server_negotiation.cc:299] Negotiation successful
0927 18:42:43.676474 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":69,"threads_started":1}
I20260927 18:42:43.677186 28863 tablet_copy_service.cc:140] P 3e5984283f154ce896d6ade588666522: Received BeginTabletCopySession request for tablet 9236b4a5871c4931ad266f5d08135454 from peer 5bcf0be3cf514817ad9a3ffe28564c37 ({username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:55219)
I20260927 18:42:43.677263 28863 tablet_copy_service.cc:161] P 3e5984283f154ce896d6ade588666522: Beginning new tablet copy session on tablet 9236b4a5871c4931ad266f5d08135454 from peer 5bcf0be3cf514817ad9a3ffe28564c37 at {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:55219: session id = 5bcf0be3cf514817ad9a3ffe28564c37-9236b4a5871c4931ad266f5d08135454
I20260927 18:42:43.677994 28863 tablet_copy_source_session.cc:205] T 9236b4a5871c4931ad266f5d08135454 P 3e5984283f154ce896d6ade588666522: Tablet Copy: opened 0 blocks and 1 log segments
I20260927 18:42:43.679283 29197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9236b4a5871c4931ad266f5d08135454. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:42:43.682610 29197 tablet_copy_client.cc:791] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: tablet copy: Starting download of 0 data blocks...
I20260927 18:42:43.682814 29197 tablet_copy_client.cc:655] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: tablet copy: Starting download of 1 WAL segments...
I20260927 18:42:43.683938 29197 tablet_copy_client.cc:531] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260927 18:42:43.684826 29197 tablet_bootstrap.cc:492] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: Bootstrap starting.
I20260927 18:42:43.686658 29197 log.cc:824] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:43.689383 29197 tablet_bootstrap.cc:492] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:42:43.689710 29197 tablet_bootstrap.cc:492] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: Bootstrap complete.
I20260927 18:42:43.689893 29197 ts_tablet_manager.cc:1397] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.003s
I20260927 18:42:43.690946 29197 raft_consensus.cc:348] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:42:43.691123 29197 raft_consensus.cc:729] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bcf0be3cf514817ad9a3ffe28564c37, State: Initialized, Role: FOLLOWER
I20260927 18:42:43.691278 29197 consensus_queue.cc:260] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d2806f483c04b6592f08e3bb1f7f62d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 34265 } } peers { permanent_uuid: "5bcf0be3cf514817ad9a3ffe28564c37" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 38775 } } peers { permanent_uuid: "3e5984283f154ce896d6ade588666522" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46563 } }
I20260927 18:42:43.691594 29197 ts_tablet_manager.cc:1428] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260927 18:42:43.691959 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:43.691800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:47333)
0927 18:42:43.691856 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:43.691914 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260927 18:42:43.692062 28863 tablet_copy_service.cc:342] P 3e5984283f154ce896d6ade588666522: Request end of tablet copy session 5bcf0be3cf514817ad9a3ffe28564c37-9236b4a5871c4931ad266f5d08135454 received from {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:55219
I20260927 18:42:43.692157 28863 tablet_copy_service.cc:433] P 3e5984283f154ce896d6ade588666522: ending tablet copy session 5bcf0be3cf514817ad9a3ffe28564c37-9236b4a5871c4931ad266f5d08135454 on tablet 9236b4a5871c4931ad266f5d08135454 with peer 5bcf0be3cf514817ad9a3ffe28564c37
I20260927 18:42:43.692334 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:43.692215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:45657)
0927 18:42:43.692265 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:43.692301 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260927 18:42:44.094677 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:44.094316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:42853)
0927 18:42:44.094515 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:44.094591 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":108,"threads_started":1}
I20260927 18:42:44.094866 29203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:44.094595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:45665)
0927 18:42:44.094740 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:44.094780 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260927 18:42:44.102931 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:44.102779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:56613)
0927 18:42:44.102865 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:44.102886 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260927 18:42:44.103055 29130 raft_consensus.cc:1203] T 9236b4a5871c4931ad266f5d08135454 P 5bcf0be3cf514817ad9a3ffe28564c37 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260927 18:42:44.103657 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:44.103514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:34223)
0927 18:42:44.103580 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:44.103618 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 18:42:45.095691 29205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:45.095379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:58787)
0927 18:42:45.095556 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:45.095615 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":72,"threads_started":1}
I20260927 18:42:45.103750 29206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:45.103498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:48727)
0927 18:42:45.103659 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:45.103700 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":110,"threads_started":1}
I20260927 18:42:45.104410 29207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:45.104068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:42723)
0927 18:42:45.104309 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:45.104361 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":67,"threads_started":1}
I20260927 18:42:46.096599 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:46.096306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:57741)
0927 18:42:46.096451 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:46.096491 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":69,"threads_started":1}
I20260927 18:42:46.104458 29209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:46.104212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:41421)
0927 18:42:46.104368 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:46.104406 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":103,"threads_started":1}
I20260927 18:42:46.105084 29210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:46.104857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:51949)
0927 18:42:46.105010 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:46.105049 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
I20260927 18:42:47.097756 29211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:47.097323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:34049)
0927 18:42:47.097564 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:47.097703 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":141,"threads_started":1}
I20260927 18:42:47.105418 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:47.105065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:46225)
0927 18:42:47.105237 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:47.105330 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":73,"threads_started":1}
I20260927 18:42:47.105921 29213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:47.105709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:34883)
0927 18:42:47.105839 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:47.105875 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":73,"threads_started":1}
I20260927 18:42:48.098819 29214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:48.098472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:39159)
0927 18:42:48.098647 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:48.098702 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":82,"threads_started":1}
I20260927 18:42:48.106374 29215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:48.105981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:45773)
0927 18:42:48.106249 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:48.106319 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":98,"threads_started":1}
I20260927 18:42:48.106663 29216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:48.106405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:51073)
0927 18:42:48.106577 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:48.106607 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":77,"threads_started":1}
I20260927 18:42:49.099701 29217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:49.099424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:54089)
0927 18:42:49.099577 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:49.099635 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
I20260927 18:42:49.107324 29218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:49.106973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:40193)
0927 18:42:49.107185 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:49.107247 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":118,"threads_started":1}
I20260927 18:42:49.107545 29219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:49.107262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:59365)
0927 18:42:49.107415 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:49.107477 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":73,"threads_started":1}
I20260927 18:42:50.100555 29220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:50.100289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:53341)
0927 18:42:50.100460 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:50.100505 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
I20260927 18:42:50.108227 29221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:50.107886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:51899)
0927 18:42:50.108081 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:50.108145 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":61,"threads_started":1}
I20260927 18:42:50.108466 29222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:50.108114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:43859)
0927 18:42:50.108338 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:50.108404 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":88,"threads_started":1}
I20260927 18:42:51.101424 29223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:51.101131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:60471)
0927 18:42:51.101299 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:51.101357 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":82,"threads_started":1}
I20260927 18:42:51.109098 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:51.108795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:40925)
0927 18:42:51.108966 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:51.109031 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":77,"threads_started":1}
I20260927 18:42:51.109279 29225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:51.108974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:38315)
0927 18:42:51.109143 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:51.109184 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260927 18:42:52.102449 29226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:52.102145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:55997)
0927 18:42:52.102333 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:52.102397 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":82,"threads_started":1}
I20260927 18:42:52.110050 29227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:52.109672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:57363)
0927 18:42:52.109901 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:52.109969 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":87,"threads_started":1}
I20260927 18:42:52.110133 29228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:52.109920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:48683)
0927 18:42:52.110033 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:52.110076 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":65,"threads_started":1}
I20260927 18:42:53.103498 29229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:53.103107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:40171)
0927 18:42:53.103332 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:53.103432 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":112,"threads_started":1}
I20260927 18:42:53.111017 29230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:53.110773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:60157)
0927 18:42:53.110919 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:53.110965 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":90,"threads_started":1}
I20260927 18:42:53.111035 29231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:53.110780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:57457)
0927 18:42:53.110939 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:53.110980 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":105,"threads_started":1}
I20260927 18:42:54.104456 29232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:54.104098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:39703)
0927 18:42:54.104323 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:54.104389 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":69,"threads_started":1}
I20260927 18:42:54.111919 29234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:54.111639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:51393)
0927 18:42:54.111816 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:54.111858 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":101,"threads_started":1}
I20260927 18:42:54.111954 29233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:54.111639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:33207)
0927 18:42:54.111854 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:54.111894 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":103,"threads_started":1}
I20260927 18:42:55.105377 29235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:55.105098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:44463)
0927 18:42:55.105271 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:55.105325 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":80,"threads_started":1}
I20260927 18:42:55.112748 29236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:55.112472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:54479)
0927 18:42:55.112633 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:55.112690 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":66,"threads_started":1}
I20260927 18:42:55.112769 29237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:55.112507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:56937)
0927 18:42:55.112684 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:55.112726 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":105,"threads_started":1}
I20260927 18:42:56.106340 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:56.106017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.194:40215)
0927 18:42:56.106202 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:56.106276 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":87,"threads_started":1}
I20260927 18:42:56.113679 29240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:56.113323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.195:57243)
0927 18:42:56.113553 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:56.113618 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":142,"threads_started":1}
I20260927 18:42:56.113747 29239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:56.113343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41973 (local address 127.11.251.193:44361)
0927 18:42:56.113643 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:56.113690 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.254:41973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":73,"threads_started":1}
I20260927 18:42:56.337159 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 28743
I20260927 18:42:56.343219 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 28912
I20260927 18:42:56.348910 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 29047
2026-09-27T18:42:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58590 ms)
[----------] 30 tests from SecurityITest (247258 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:42:56 dist-test-slave-z47j krb5kdc[29243](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:42:56 dist-test-slave-z47j krb5kdc[29243](info): set up 2 sockets
Sep 27 18:42:56 dist-test-slave-z47j krb5kdc[29243](info): commencing operation
krb5kdc: starting...
W20260927 18:42:57.435141 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:42:57 dist-test-slave-z47j krb5kdc[29243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:42:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:42:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:42:57.570171 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:37679
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:36099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:37679
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:42:57.651183 29256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:57.651387 29256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:57.651409 29256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:57.652518 29256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:42:57.652558 29256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:57.652572 29256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:57.652585 29256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:42:57.652597 29256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:42:57.654012 29256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:36099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:37679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:37679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:42:57.654207 29256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:57.654373 29256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:57.660022 29264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:57.660066 29262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:57.660022 29261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:57.660457 29256 server_base.cc:1034] running on GCE node
I20260927 18:42:57.660768 29256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:57.661127 29256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:57.662312 29256 hybrid_clock.cc:648] HybridClock initialized: now 1790534577662279 us; error 52 us; skew 500 ppm
Sep 27 18:42:57 dist-test-slave-z47j krb5kdc[29243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534577, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:42:57.665151 29256 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:42:57.665735 29256 webserver.cc:460] Webserver started at http://127.11.251.254:41013/ using document root <none> and password file <none>
I20260927 18:42:57.665998 29256 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:57.666049 29256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:57.666215 29256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:42:57.667452 29256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "1587608a19a44bcdbd1d282a23274030"
format_stamp: "Formatted at 2026-09-27 18:42:57 on dist-test-slave-z47j"
server_key: "73139339e7c3d0eccc772b3eba6a2555"
server_key_iv: "7f65e182a35ac0b523b949ad72e19100"
server_key_version: "encryptionkey@0"
I20260927 18:42:57.667848 29256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "1587608a19a44bcdbd1d282a23274030"
format_stamp: "Formatted at 2026-09-27 18:42:57 on dist-test-slave-z47j"
server_key: "73139339e7c3d0eccc772b3eba6a2555"
server_key_iv: "7f65e182a35ac0b523b949ad72e19100"
server_key_version: "encryptionkey@0"
I20260927 18:42:57.669940 29256 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260927 18:42:57.670861 29271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:57.671129 29256 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:42:57.671221 29256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "1587608a19a44bcdbd1d282a23274030"
format_stamp: "Formatted at 2026-09-27 18:42:57 on dist-test-slave-z47j"
server_key: "73139339e7c3d0eccc772b3eba6a2555"
server_key_iv: "7f65e182a35ac0b523b949ad72e19100"
server_key_version: "encryptionkey@0"
I20260927 18:42:57.671329 29256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:57.694792 29256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:57.697101 29256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:57.697271 29256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:57.702221 29256 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:37679
I20260927 18:42:57.702365 29323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:37679 every 8 connection(s)
I20260927 18:42:57.702736 29256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:42:57.703864 29324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:42:57.705379 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 29256
I20260927 18:42:57.705483 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:42:57.705770 12271 external_mini_cluster.cc:1428] Setting key 5939b913cde9fac6e65d011490400f7f
I20260927 18:42:57.706727 29324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030: Bootstrap starting.
I20260927 18:42:57.707363 29324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030: Neither blocks nor log segments found. Creating new log.
I20260927 18:42:57.707743 29324 log.cc:824] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:57.708613 29324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030: No bootstrap required, opened a new log
I20260927 18:42:57.711126 29324 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1587608a19a44bcdbd1d282a23274030" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37679 } }
I20260927 18:42:57.711334 29324 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:42:57.711382 29324 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1587608a19a44bcdbd1d282a23274030, State: Initialized, Role: FOLLOWER
I20260927 18:42:57.711542 29324 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1587608a19a44bcdbd1d282a23274030" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37679 } }
I20260927 18:42:57.711642 29324 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:42:57.711673 29324 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:42:57.711726 29324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:42:57.712535 29324 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1587608a19a44bcdbd1d282a23274030" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37679 } }
I20260927 18:42:57.712668 29324 leader_election.cc:302] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1587608a19a44bcdbd1d282a23274030; no voters:
I20260927 18:42:57.712894 29324 leader_election.cc:288] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:42:57.712972 29329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:42:57.713184 29324 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:42:57.713184 29329 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030 [term 1 LEADER]: Becoming Leader. State: Replica: 1587608a19a44bcdbd1d282a23274030, State: Running, Role: LEADER
I20260927 18:42:57.713398 29329 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1587608a19a44bcdbd1d282a23274030" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37679 } }
I20260927 18:42:57.713798 29329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1587608a19a44bcdbd1d282a23274030. Latest consensus state: current_term: 1 leader_uuid: "1587608a19a44bcdbd1d282a23274030" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1587608a19a44bcdbd1d282a23274030" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37679 } } }
Sep 27 18:42:57 dist-test-slave-z47j krb5kdc[29243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:42:57.713884 29329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030 [sys.catalog]: This master's current role is: LEADER
I20260927 18:42:57.713981 29331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1587608a19a44bcdbd1d282a23274030" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1587608a19a44bcdbd1d282a23274030" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37679 } } }
I20260927 18:42:57.714658 29331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030 [sys.catalog]: This master's current role is: LEADER
I20260927 18:42:57.714133 29333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:42:57.715569 29333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:42:57.718462 29333 catalog_manager.cc:1347] Generated new cluster ID: e49f01cf894e4bd5880cb06f187a5fb6
I20260927 18:42:57.718519 29333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:42:57.720072 29327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:57.706459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49930 (local address 127.11.251.254:37679)
0927 18:42:57.706679 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:57.706688 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:57.706720 (+ 32us) server_negotiation.cc:407] Connection header received
0927 18:42:57.707339 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:57.707347 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:57.707405 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:57.707597 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:57.708425 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:57.709471 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:57.710700 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:57.711181 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:57.711276 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:57.714429 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:57.714434 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:57.714434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:57.714448 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:57.718029 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:57.718506 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:57.718510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:57.718510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:57.718671 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:57.718981 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:57.718982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:57.718982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:57.719324 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:57.719550 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:57.719628 (+ 78us) server_negotiation.cc:299] Negotiation successful
0927 18:42:57.719818 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":120,"threads_started":1}
I20260927 18:42:57.742849 29333 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:42:57.743685 29333 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:42:57.756115 29333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030: Generated new TSK 0
I20260927 18:42:57.756343 29333 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.251.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
I20260927 18:42:57.793707 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:37679
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:36099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:42:57.885214 29350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:57.885452 29350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:57.885479 29350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:57.886837 29350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:57.886891 29350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:57.886926 29350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:42:57.888583 29350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:36099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:37679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:42:57.888867 29350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:57.889164 29350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:57.894728 29355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:57.894737 29358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:57.894740 29356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:57.895126 29350 server_base.cc:1034] running on GCE node
I20260927 18:42:57.895373 29350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:57.895654 29350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:57.896864 29350 hybrid_clock.cc:648] HybridClock initialized: now 1790534577896848 us; error 57 us; skew 500 ppm
Sep 27 18:42:57 dist-test-slave-z47j krb5kdc[29243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534577, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:42:57.899854 29350 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:42:57.900401 29350 webserver.cc:460] Webserver started at http://127.11.251.193:41199/ using document root <none> and password file <none>
I20260927 18:42:57.900671 29350 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:57.900743 29350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:57.900902 29350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:42:57.902089 29350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "50a58b48e19943c5a8bdcefe69d60022"
format_stamp: "Formatted at 2026-09-27 18:42:57 on dist-test-slave-z47j"
server_key: "6e000d78128de80e63bad441a278d6f8"
server_key_iv: "418e5d000df8974f004f8085e2e212f9"
server_key_version: "encryptionkey@0"
I20260927 18:42:57.902489 29350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "50a58b48e19943c5a8bdcefe69d60022"
format_stamp: "Formatted at 2026-09-27 18:42:57 on dist-test-slave-z47j"
server_key: "6e000d78128de80e63bad441a278d6f8"
server_key_iv: "418e5d000df8974f004f8085e2e212f9"
server_key_version: "encryptionkey@0"
I20260927 18:42:57.904018 29350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:42:57.905040 29365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:57.905310 29350 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:42:57.905417 29350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "50a58b48e19943c5a8bdcefe69d60022"
format_stamp: "Formatted at 2026-09-27 18:42:57 on dist-test-slave-z47j"
server_key: "6e000d78128de80e63bad441a278d6f8"
server_key_iv: "418e5d000df8974f004f8085e2e212f9"
server_key_version: "encryptionkey@0"
I20260927 18:42:57.905509 29350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:57.914856 29350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:57.917115 29350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:57.917306 29350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:57.917555 29350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:42:57.917972 29350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:42:57.918031 29350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:57.918076 29350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:42:57.918107 29350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:57.924914 29350 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:46845
I20260927 18:42:57.924971 29478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:46845 every 8 connection(s)
I20260927 18:42:57.925313 29350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:42:57.929818 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 29350
I20260927 18:42:57.929912 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:42:57.930141 12271 external_mini_cluster.cc:1428] Setting key 442a275238a7c2244990fe6b8852fcd2
Sep 27 18:42:57 dist-test-slave-z47j krb5kdc[29243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534577, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:42:57.937143 29327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:57.926212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:50283 (local address 127.11.251.254:37679)
0927 18:42:57.926304 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:57.926307 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:57.926549 (+ 242us) server_negotiation.cc:407] Connection header received
0927 18:42:57.927141 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:57.927142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:57.927153 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:57.927220 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:57.928554 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:57.929366 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:57.930823 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:57.931102 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:57.931199 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:57.934072 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:57.934073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:57.934074 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:57.934082 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:57.935649 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:57.936100 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:57.936101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:57.936102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:57.936161 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:57.936415 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:57.936416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:57.936417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:57.936552 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:57.936624 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:57.936925 (+ 301us) server_negotiation.cc:299] Negotiation successful
0927 18:42:57.937036 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:42:57.937350 29481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:57.926172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:37679 (local address 127.11.251.193:50283)
0927 18:42:57.926380 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:57.926426 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:57.927042 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:57.927289 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:57.927292 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:57.928044 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:57.928432 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:57.928439 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:57.929461 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:57.929462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:57.930711 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:57.930714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:57.931209 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:57.931210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:57.931356 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:57.932027 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:57.932056 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:57.933950 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:57.935751 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:57.935759 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:57.935761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:57.936006 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:57.936230 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:57.936231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:57.936232 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:57.936345 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:57.936624 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:57.936632 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:57.936784 (+ 152us) client_negotiation.cc:769] Sending connection context
0927 18:42:57.936895 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 18:42:57.937074 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":113,"threads_started":1}
I20260927 18:42:57.938014 29479 heartbeater.cc:347] Connected to a master server at 127.11.251.254:37679
I20260927 18:42:57.938124 29479 heartbeater.cc:464] Registering TS with master...
I20260927 18:42:57.938347 29479 heartbeater.cc:511] Master 127.11.251.254:37679 requested a full tablet report, sending...
I20260927 18:42:57.938946 29288 ts_manager.cc:194] Registered new tserver with Master: 50a58b48e19943c5a8bdcefe69d60022 (127.11.251.193:46845)
WARNING: no policy specified for kudu/127.11.251.194@KRBTEST.COM; defaulting to no policy
I20260927 18:42:57.939709 29288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:50283
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
I20260927 18:42:57.961830 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:37679
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:36099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:42:58.048985 29484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:58.049260 29484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:58.049304 29484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:58.050479 29484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:58.050551 29484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:58.050597 29484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:42:58.052083 29484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:36099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:37679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:42:58.052363 29484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:58.052609 29484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:58.057957 29489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:58.057986 29490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:58.058128 29484 server_base.cc:1034] running on GCE node
W20260927 18:42:58.057986 29492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:58.058418 29484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:58.058631 29484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:58.059782 29484 hybrid_clock.cc:648] HybridClock initialized: now 1790534578059754 us; error 50 us; skew 500 ppm
Sep 27 18:42:58 dist-test-slave-z47j krb5kdc[29243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534578, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:42:58.062469 29484 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:42:58.062963 29484 webserver.cc:460] Webserver started at http://127.11.251.194:39487/ using document root <none> and password file <none>
I20260927 18:42:58.063211 29484 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:58.063264 29484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:58.063405 29484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:42:58.064504 29484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "8628587183e141879420c17baaca6bcd"
format_stamp: "Formatted at 2026-09-27 18:42:58 on dist-test-slave-z47j"
server_key: "984dc6a59ed0c9610c4748377a51d53a"
server_key_iv: "3418e8d2d1104668eb17618aea20016e"
server_key_version: "encryptionkey@0"
I20260927 18:42:58.064881 29484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "8628587183e141879420c17baaca6bcd"
format_stamp: "Formatted at 2026-09-27 18:42:58 on dist-test-slave-z47j"
server_key: "984dc6a59ed0c9610c4748377a51d53a"
server_key_iv: "3418e8d2d1104668eb17618aea20016e"
server_key_version: "encryptionkey@0"
I20260927 18:42:58.066499 29484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 18:42:58.067437 29499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:58.067691 29484 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:42:58.067831 29484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "8628587183e141879420c17baaca6bcd"
format_stamp: "Formatted at 2026-09-27 18:42:58 on dist-test-slave-z47j"
server_key: "984dc6a59ed0c9610c4748377a51d53a"
server_key_iv: "3418e8d2d1104668eb17618aea20016e"
server_key_version: "encryptionkey@0"
I20260927 18:42:58.067950 29484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:58.099175 29484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:58.101871 29484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:58.102005 29484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:58.102245 29484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:42:58.102622 29484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:42:58.102659 29484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:58.102685 29484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:42:58.102699 29484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:58.110127 29484 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:45607
I20260927 18:42:58.110248 29612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:45607 every 8 connection(s)
I20260927 18:42:58.110602 29484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
Sep 27 18:42:58 dist-test-slave-z47j krb5kdc[29243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534578, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:42:58.118809 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 29484
I20260927 18:42:58.118988 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:42:58.119257 12271 external_mini_cluster.cc:1428] Setting key b267ec8fb4fae34b266d621d507bff10
I20260927 18:42:58.121862 29615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:58.111469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:37679 (local address 127.11.251.194:41409)
0927 18:42:58.111666 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:58.111711 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:58.112211 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:58.112479 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:58.112483 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:58.113084 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:58.113410 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:58.113417 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.114519 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.114521 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:58.115538 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:58.115541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.115958 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.115958 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:58.116056 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:58.116636 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:58.116644 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:58.118080 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:58.120407 (+ 2327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:58.120412 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:58.120414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:58.120649 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:58.120886 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:58.120887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:58.120887 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:58.120981 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:58.121289 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:58.121293 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:58.121403 (+ 110us) client_negotiation.cc:769] Sending connection context
0927 18:42:58.121491 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 18:42:58.121681 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":119,"threads_started":1}
I20260927 18:42:58.121860 29327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:58.111506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:41409 (local address 127.11.251.254:37679)
0927 18:42:58.111602 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:58.111604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:58.111872 (+ 268us) server_negotiation.cc:407] Connection header received
0927 18:42:58.112317 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:58.112318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:58.112331 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:58.112413 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:58.113554 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.114398 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.115658 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.115870 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.115943 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:58.118193 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:58.118195 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:58.118195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:58.118203 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:58.120304 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:58.120731 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:58.120732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:58.120732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:58.120808 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:58.121054 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:58.121055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:58.121055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:58.121217 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:58.121293 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:58.121539 (+ 246us) server_negotiation.cc:299] Negotiation successful
0927 18:42:58.121687 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 18:42:58.122756 29613 heartbeater.cc:347] Connected to a master server at 127.11.251.254:37679
I20260927 18:42:58.122870 29613 heartbeater.cc:464] Registering TS with master...
I20260927 18:42:58.123075 29613 heartbeater.cc:511] Master 127.11.251.254:37679 requested a full tablet report, sending...
I20260927 18:42:58.123544 29288 ts_manager.cc:194] Registered new tserver with Master: 8628587183e141879420c17baaca6bcd (127.11.251.194:45607)
I20260927 18:42:58.124104 29288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:41409
WARNING: no policy specified for kudu/127.11.251.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
I20260927 18:42:58.152343 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:37679
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:36099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:42:58.236431 29618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:58.236672 29618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:58.236716 29618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:58.237972 29618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:58.238039 29618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:58.238095 29618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:42:58.239583 29618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:36099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:37679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:42:58.239847 29618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:58.240113 29618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:58.245687 29624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:58.245775 29618 server_base.cc:1034] running on GCE node
W20260927 18:42:58.245687 29626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:58.245785 29623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:58.246264 29618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:58.246567 29618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:58.247750 29618 hybrid_clock.cc:648] HybridClock initialized: now 1790534578247735 us; error 42 us; skew 500 ppm
Sep 27 18:42:58 dist-test-slave-z47j krb5kdc[29243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534578, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:42:58.250401 29618 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:42:58.250917 29618 webserver.cc:460] Webserver started at http://127.11.251.195:36147/ using document root <none> and password file <none>
I20260927 18:42:58.251188 29618 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:58.251240 29618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:58.251376 29618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:42:58.252445 29618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "45c2f2257c16439899c1f2fcf6ea235a"
format_stamp: "Formatted at 2026-09-27 18:42:58 on dist-test-slave-z47j"
server_key: "4e7dbac9a2644d9c4ef3ef4dd0778df3"
server_key_iv: "4b40b236d2b20e52c1a537d242e88f9e"
server_key_version: "encryptionkey@0"
I20260927 18:42:58.252810 29618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "45c2f2257c16439899c1f2fcf6ea235a"
format_stamp: "Formatted at 2026-09-27 18:42:58 on dist-test-slave-z47j"
server_key: "4e7dbac9a2644d9c4ef3ef4dd0778df3"
server_key_iv: "4b40b236d2b20e52c1a537d242e88f9e"
server_key_version: "encryptionkey@0"
I20260927 18:42:58.254406 29618 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260927 18:42:58.255532 29633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:58.255877 29618 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:42:58.255973 29618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "45c2f2257c16439899c1f2fcf6ea235a"
format_stamp: "Formatted at 2026-09-27 18:42:58 on dist-test-slave-z47j"
server_key: "4e7dbac9a2644d9c4ef3ef4dd0778df3"
server_key_iv: "4b40b236d2b20e52c1a537d242e88f9e"
server_key_version: "encryptionkey@0"
I20260927 18:42:58.256069 29618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:58.274700 29618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:58.276871 29618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:58.277019 29618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:58.277295 29618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:42:58.277690 29618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:42:58.277748 29618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:58.277783 29618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:42:58.277810 29618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:58.284766 29618 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:42901
I20260927 18:42:58.284889 29746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:42901 every 8 connection(s)
I20260927 18:42:58.285190 29618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:42:58.288269 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 29618
I20260927 18:42:58.288367 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:42:58.288586 12271 external_mini_cluster.cc:1428] Setting key 645790e3884e67b664d9c567fa5da7d9
Sep 27 18:42:58 dist-test-slave-z47j krb5kdc[29243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534578, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:42:58.296617 29327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:58.286168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:42155 (local address 127.11.251.254:37679)
0927 18:42:58.286281 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:58.286285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:58.286535 (+ 250us) server_negotiation.cc:407] Connection header received
0927 18:42:58.287214 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:58.287216 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:58.287231 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:58.287306 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:58.288566 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.289250 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.290526 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.290770 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.290875 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:58.293418 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:58.293421 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:58.293421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:58.293431 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:58.295106 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:58.295610 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:58.295611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:58.295611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:58.295669 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:58.295950 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:58.295951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:58.295952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:58.296087 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:58.296158 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:58.296461 (+ 303us) server_negotiation.cc:299] Negotiation successful
0927 18:42:58.296536 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260927 18:42:58.296862 29749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:58.286080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:37679 (local address 127.11.251.195:42155)
0927 18:42:58.286328 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:58.286388 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:58.287082 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:58.287407 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:58.287414 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:58.288136 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:58.288452 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:58.288460 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.289376 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.289377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:58.290415 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:58.290417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.290888 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.290889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:58.291043 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:58.291644 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:58.291654 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:58.293276 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:58.295234 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:58.295241 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:58.295244 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:58.295521 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:58.295745 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:58.295745 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:58.295746 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:58.295853 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:58.296164 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:58.296168 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:58.296324 (+ 156us) client_negotiation.cc:769] Sending connection context
0927 18:42:58.296417 (+ 93us) client_negotiation.cc:240] Negotiation successful
0927 18:42:58.296560 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":93,"threads_started":1}
I20260927 18:42:58.297466 29747 heartbeater.cc:347] Connected to a master server at 127.11.251.254:37679
I20260927 18:42:58.297668 29747 heartbeater.cc:464] Registering TS with master...
I20260927 18:42:58.297959 29747 heartbeater.cc:511] Master 127.11.251.254:37679 requested a full tablet report, sending...
I20260927 18:42:58.298497 29288 ts_manager.cc:194] Registered new tserver with Master: 45c2f2257c16439899c1f2fcf6ea235a (127.11.251.195:42901)
I20260927 18:42:58.298923 29288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:42155
I20260927 18:42:58.300401 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:42:58.308918 29327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:58.301984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49938 (local address 127.11.251.254:37679)
0927 18:42:58.302064 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:58.302066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:58.302082 (+ 16us) server_negotiation.cc:407] Connection header received
0927 18:42:58.302171 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:58.302171 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:58.302181 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:58.302253 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:58.302828 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.303375 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.304125 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.304328 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.304394 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:58.305767 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:58.305768 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:58.305769 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:58.305776 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:58.307387 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:58.307766 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:58.307767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:58.307767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:58.307824 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:58.308178 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:58.308180 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:58.308180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:58.308367 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:58.308548 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:58.308700 (+ 152us) server_negotiation.cc:299] Negotiation successful
0927 18:42:58.308795 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:42:58.311044 29288 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49938:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:42:58.313254 29288 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:42:58.321292 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:58.316648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:46845 (local address 127.0.0.1:52130)
0927 18:42:58.317322 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:58.317334 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:58.317386 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:58.317705 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:58.317707 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:58.317709 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:42:58.317752 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:58.317754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.318999 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.319000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:58.319928 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:58.319931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.320893 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.320895 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:58.320996 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:58.321015 (+ 19us) client_negotiation.cc:769] Sending connection context
0927 18:42:58.321103 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 18:42:58.321181 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"mutex_wait_us":2,"thread_start_us":100,"threads_started":1}
I20260927 18:42:58.321293 29758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:58.316418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:45607 (local address 127.0.0.1:43962)
0927 18:42:58.316541 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:58.316582 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:58.316659 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:58.317307 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:58.317308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:58.317311 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:42:58.317367 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:58.317368 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.318547 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.318548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:58.320116 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:58.320118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.321025 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.321026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:58.321106 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:58.321111 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:42:58.321160 (+ 49us) client_negotiation.cc:240] Negotiation successful
0927 18:42:58.321196 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":13,"thread_start_us":112,"threads_started":1}
I20260927 18:42:58.321521 29762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:58.316834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43962 (local address 127.11.251.194:45607)
0927 18:42:58.317044 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:58.317048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:58.317060 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:42:58.317107 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:58.317109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:58.317128 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:58.317216 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:42:58.317429 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.318410 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.320234 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.320806 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.320917 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:58.320986 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:58.321337 (+ 351us) server_negotiation.cc:299] Negotiation successful
0927 18:42:58.321397 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":112,"threads_started":1}
I20260927 18:42:58.321627 29763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:58.317145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52130 (local address 127.11.251.193:46845)
0927 18:42:58.317500 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:58.317506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:58.317516 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:42:58.317559 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:58.317560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:58.317613 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:58.317702 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:42:58.317835 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.318882 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.320027 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.320777 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.320864 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:58.320925 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:58.321136 (+ 211us) server_negotiation.cc:299] Negotiation successful
0927 18:42:58.321475 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":82,"threads_started":1}
I20260927 18:42:58.322188 29760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:58.316584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40346 (local address 127.11.251.195:42901)
0927 18:42:58.316844 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:58.316850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:58.316862 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:42:58.316913 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:58.316916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:58.316946 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:58.317049 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:42:58.317380 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.318591 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.320217 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.320890 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.320957 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:58.320999 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:58.321344 (+ 345us) server_negotiation.cc:299] Negotiation successful
0927 18:42:58.322099 (+ 755us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":123,"threads_started":1}
I20260927 18:42:58.322499 29759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:58.316420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:42901 (local address 127.0.0.1:40346)
0927 18:42:58.316642 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:58.316658 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:58.316726 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:58.317166 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:58.317170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:58.317174 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:42:58.317262 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:58.317264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.318733 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.318734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:58.320115 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:58.320118 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.321047 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.321048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:58.321154 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:58.321160 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:42:58.321213 (+ 53us) client_negotiation.cc:240] Negotiation successful
0927 18:42:58.321291 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":41,"thread_start_us":65,"threads_started":1}
I20260927 18:42:58.322450 29547 tablet_service.cc:1467] Processing CreateTablet for tablet 5b7f1af070bc43b7b322d459f16a8a6e (DEFAULT_TABLE table=test-table [id=aff37b09c67646dd8ffb2dfbbc39078c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:42:58.322469 29413 tablet_service.cc:1467] Processing CreateTablet for tablet 5b7f1af070bc43b7b322d459f16a8a6e (DEFAULT_TABLE table=test-table [id=aff37b09c67646dd8ffb2dfbbc39078c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:42:58.322875 29547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b7f1af070bc43b7b322d459f16a8a6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:42:58.322903 29413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b7f1af070bc43b7b322d459f16a8a6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:42:58.323454 29681 tablet_service.cc:1467] Processing CreateTablet for tablet 5b7f1af070bc43b7b322d459f16a8a6e (DEFAULT_TABLE table=test-table [id=aff37b09c67646dd8ffb2dfbbc39078c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:42:58.323761 29681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b7f1af070bc43b7b322d459f16a8a6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:42:58.326355 29764 tablet_bootstrap.cc:492] T 5b7f1af070bc43b7b322d459f16a8a6e P 50a58b48e19943c5a8bdcefe69d60022: Bootstrap starting.
I20260927 18:42:58.326650 29765 tablet_bootstrap.cc:492] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd: Bootstrap starting.
I20260927 18:42:58.327119 29766 tablet_bootstrap.cc:492] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a: Bootstrap starting.
I20260927 18:42:58.327373 29765 tablet_bootstrap.cc:654] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd: Neither blocks nor log segments found. Creating new log.
I20260927 18:42:58.327574 29764 tablet_bootstrap.cc:654] T 5b7f1af070bc43b7b322d459f16a8a6e P 50a58b48e19943c5a8bdcefe69d60022: Neither blocks nor log segments found. Creating new log.
I20260927 18:42:58.327950 29766 tablet_bootstrap.cc:654] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a: Neither blocks nor log segments found. Creating new log.
I20260927 18:42:58.328032 29765 log.cc:824] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:58.328305 29764 log.cc:824] T 5b7f1af070bc43b7b322d459f16a8a6e P 50a58b48e19943c5a8bdcefe69d60022: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:58.328679 29766 log.cc:824] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:58.328868 29765 tablet_bootstrap.cc:492] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd: No bootstrap required, opened a new log
I20260927 18:42:58.328951 29765 ts_tablet_manager.cc:1397] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:42:58.329372 29764 tablet_bootstrap.cc:492] T 5b7f1af070bc43b7b322d459f16a8a6e P 50a58b48e19943c5a8bdcefe69d60022: No bootstrap required, opened a new log
I20260927 18:42:58.329481 29764 ts_tablet_manager.cc:1397] T 5b7f1af070bc43b7b322d459f16a8a6e P 50a58b48e19943c5a8bdcefe69d60022: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:42:58.329623 29766 tablet_bootstrap.cc:492] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a: No bootstrap required, opened a new log
I20260927 18:42:58.329713 29766 ts_tablet_manager.cc:1397] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:42:58.331029 29765 raft_consensus.cc:348] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c2f2257c16439899c1f2fcf6ea235a" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 42901 } } peers { permanent_uuid: "8628587183e141879420c17baaca6bcd" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 45607 } } peers { permanent_uuid: "50a58b48e19943c5a8bdcefe69d60022" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46845 } }
I20260927 18:42:58.331248 29765 raft_consensus.cc:374] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:42:58.331288 29765 raft_consensus.cc:729] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8628587183e141879420c17baaca6bcd, State: Initialized, Role: FOLLOWER
I20260927 18:42:58.331171 29764 raft_consensus.cc:348] T 5b7f1af070bc43b7b322d459f16a8a6e P 50a58b48e19943c5a8bdcefe69d60022 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c2f2257c16439899c1f2fcf6ea235a" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 42901 } } peers { permanent_uuid: "8628587183e141879420c17baaca6bcd" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 45607 } } peers { permanent_uuid: "50a58b48e19943c5a8bdcefe69d60022" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46845 } }
I20260927 18:42:58.331378 29764 raft_consensus.cc:374] T 5b7f1af070bc43b7b322d459f16a8a6e P 50a58b48e19943c5a8bdcefe69d60022 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:42:58.331418 29764 raft_consensus.cc:729] T 5b7f1af070bc43b7b322d459f16a8a6e P 50a58b48e19943c5a8bdcefe69d60022 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50a58b48e19943c5a8bdcefe69d60022, State: Initialized, Role: FOLLOWER
I20260927 18:42:58.331463 29765 consensus_queue.cc:260] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c2f2257c16439899c1f2fcf6ea235a" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 42901 } } peers { permanent_uuid: "8628587183e141879420c17baaca6bcd" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 45607 } } peers { permanent_uuid: "50a58b48e19943c5a8bdcefe69d60022" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46845 } }
I20260927 18:42:58.331586 29764 consensus_queue.cc:260] T 5b7f1af070bc43b7b322d459f16a8a6e P 50a58b48e19943c5a8bdcefe69d60022 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c2f2257c16439899c1f2fcf6ea235a" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 42901 } } peers { permanent_uuid: "8628587183e141879420c17baaca6bcd" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 45607 } } peers { permanent_uuid: "50a58b48e19943c5a8bdcefe69d60022" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46845 } }
I20260927 18:42:58.331821 29765 ts_tablet_manager.cc:1428] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:42:58.331746 29766 raft_consensus.cc:348] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c2f2257c16439899c1f2fcf6ea235a" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 42901 } } peers { permanent_uuid: "8628587183e141879420c17baaca6bcd" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 45607 } } peers { permanent_uuid: "50a58b48e19943c5a8bdcefe69d60022" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46845 } }
I20260927 18:42:58.331933 29764 ts_tablet_manager.cc:1428] T 5b7f1af070bc43b7b322d459f16a8a6e P 50a58b48e19943c5a8bdcefe69d60022: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260927 18:42:58.331945 29766 raft_consensus.cc:374] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:42:58.331990 29766 raft_consensus.cc:729] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c2f2257c16439899c1f2fcf6ea235a, State: Initialized, Role: FOLLOWER
I20260927 18:42:58.332103 29479 heartbeater.cc:503] Master 127.11.251.254:37679 was elected leader, sending a full tablet report...
I20260927 18:42:58.332113 29613 heartbeater.cc:503] Master 127.11.251.254:37679 was elected leader, sending a full tablet report...
I20260927 18:42:58.332108 29766 consensus_queue.cc:260] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c2f2257c16439899c1f2fcf6ea235a" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 42901 } } peers { permanent_uuid: "8628587183e141879420c17baaca6bcd" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 45607 } } peers { permanent_uuid: "50a58b48e19943c5a8bdcefe69d60022" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46845 } }
I20260927 18:42:58.332427 29766 ts_tablet_manager.cc:1428] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260927 18:42:58.332479 29747 heartbeater.cc:503] Master 127.11.251.254:37679 was elected leader, sending a full tablet report...
W20260927 18:42:58.361258 29614 tablet.cc:2369] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:42:58.426157 29480 tablet.cc:2369] T 5b7f1af070bc43b7b322d459f16a8a6e P 50a58b48e19943c5a8bdcefe69d60022: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:42:58.516595 29772 raft_consensus.cc:482] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:42:58.516788 29772 raft_consensus.cc:504] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c2f2257c16439899c1f2fcf6ea235a" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 42901 } } peers { permanent_uuid: "8628587183e141879420c17baaca6bcd" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 45607 } } peers { permanent_uuid: "50a58b48e19943c5a8bdcefe69d60022" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46845 } }
I20260927 18:42:58.517303 29772 leader_election.cc:288] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8628587183e141879420c17baaca6bcd (127.11.251.194:45607), 50a58b48e19943c5a8bdcefe69d60022 (127.11.251.193:46845)
I20260927 18:42:58.521997 29762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:58.517703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:53141 (local address 127.11.251.194:45607)
0927 18:42:58.517762 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:58.517766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:58.517848 (+ 82us) server_negotiation.cc:407] Connection header received
0927 18:42:58.517887 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:58.517888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:58.517923 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:58.517995 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:42:58.518338 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.519173 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.520918 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.521542 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.521663 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:58.521674 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:58.521862 (+ 188us) server_negotiation.cc:299] Negotiation successful
0927 18:42:58.521924 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:42:58.521997 29763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:58.517714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:56473 (local address 127.11.251.193:46845)
0927 18:42:58.517775 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:58.517778 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:58.517934 (+ 156us) server_negotiation.cc:407] Connection header received
0927 18:42:58.517966 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:58.517967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:58.517978 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:58.518040 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:42:58.518339 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.519173 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.520919 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.521542 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.521662 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:58.521674 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:58.521862 (+ 188us) server_negotiation.cc:299] Negotiation successful
0927 18:42:58.521910 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:42:58.522001 29749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:58.517568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:45607 (local address 127.11.251.195:53141)
0927 18:42:58.517680 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:58.517693 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:58.517756 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:58.518053 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:58.518054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:58.518057 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:42:58.518210 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:58.518211 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.519299 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.519300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:58.520784 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:58.520787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.521679 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.521680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:58.521764 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:58.521771 (+ 7us) client_negotiation.cc:769] Sending connection context
0927 18:42:58.521836 (+ 65us) client_negotiation.cc:240] Negotiation successful
0927 18:42:58.521890 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":132}
I20260927 18:42:58.522011 29773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:58.517620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:46845 (local address 127.11.251.195:56473)
0927 18:42:58.517839 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:58.517854 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:58.517910 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:58.518066 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:58.518066 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:58.518068 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:42:58.518164 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:58.518167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.519299 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.519300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:58.520784 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:58.520787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.521699 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.521700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:58.521764 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:58.521772 (+ 8us) client_negotiation.cc:769] Sending connection context
0927 18:42:58.521836 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 18:42:58.521886 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":122,"thread_start_us":140,"threads_started":1}
I20260927 18:42:58.522562 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b7f1af070bc43b7b322d459f16a8a6e" candidate_uuid: "45c2f2257c16439899c1f2fcf6ea235a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8628587183e141879420c17baaca6bcd" is_pre_election: true
I20260927 18:42:58.522795 29567 raft_consensus.cc:2454] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45c2f2257c16439899c1f2fcf6ea235a in term 0.
I20260927 18:42:58.522746 29433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b7f1af070bc43b7b322d459f16a8a6e" candidate_uuid: "45c2f2257c16439899c1f2fcf6ea235a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50a58b48e19943c5a8bdcefe69d60022" is_pre_election: true
I20260927 18:42:58.522895 29433 raft_consensus.cc:2454] T 5b7f1af070bc43b7b322d459f16a8a6e P 50a58b48e19943c5a8bdcefe69d60022 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45c2f2257c16439899c1f2fcf6ea235a in term 0.
I20260927 18:42:58.523101 29635 leader_election.cc:302] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c2f2257c16439899c1f2fcf6ea235a, 50a58b48e19943c5a8bdcefe69d60022; no voters:
I20260927 18:42:58.523310 29772 raft_consensus.cc:2781] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:42:58.523432 29772 raft_consensus.cc:482] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:42:58.523455 29772 raft_consensus.cc:3037] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:42:58.524633 29772 raft_consensus.cc:504] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c2f2257c16439899c1f2fcf6ea235a" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 42901 } } peers { permanent_uuid: "8628587183e141879420c17baaca6bcd" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 45607 } } peers { permanent_uuid: "50a58b48e19943c5a8bdcefe69d60022" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46845 } }
I20260927 18:42:58.524940 29772 leader_election.cc:288] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a [CANDIDATE]: Term 1 election: Requested vote from peers 8628587183e141879420c17baaca6bcd (127.11.251.194:45607), 50a58b48e19943c5a8bdcefe69d60022 (127.11.251.193:46845)
I20260927 18:42:58.525262 29433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b7f1af070bc43b7b322d459f16a8a6e" candidate_uuid: "45c2f2257c16439899c1f2fcf6ea235a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50a58b48e19943c5a8bdcefe69d60022"
I20260927 18:42:58.525414 29433 raft_consensus.cc:3037] T 5b7f1af070bc43b7b322d459f16a8a6e P 50a58b48e19943c5a8bdcefe69d60022 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:42:58.525667 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b7f1af070bc43b7b322d459f16a8a6e" candidate_uuid: "45c2f2257c16439899c1f2fcf6ea235a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8628587183e141879420c17baaca6bcd"
I20260927 18:42:58.525789 29567 raft_consensus.cc:3037] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:42:58.526552 29433 raft_consensus.cc:2454] T 5b7f1af070bc43b7b322d459f16a8a6e P 50a58b48e19943c5a8bdcefe69d60022 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45c2f2257c16439899c1f2fcf6ea235a in term 1.
I20260927 18:42:58.526791 29635 leader_election.cc:302] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c2f2257c16439899c1f2fcf6ea235a, 50a58b48e19943c5a8bdcefe69d60022; no voters:
I20260927 18:42:58.526834 29567 raft_consensus.cc:2454] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45c2f2257c16439899c1f2fcf6ea235a in term 1.
I20260927 18:42:58.526916 29772 raft_consensus.cc:2781] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:42:58.527135 29772 raft_consensus.cc:686] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a [term 1 LEADER]: Becoming Leader. State: Replica: 45c2f2257c16439899c1f2fcf6ea235a, State: Running, Role: LEADER
I20260927 18:42:58.527282 29772 consensus_queue.cc:237] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c2f2257c16439899c1f2fcf6ea235a" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 42901 } } peers { permanent_uuid: "8628587183e141879420c17baaca6bcd" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 45607 } } peers { permanent_uuid: "50a58b48e19943c5a8bdcefe69d60022" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46845 } }
I20260927 18:42:58.528440 29285 catalog_manager.cc:5495] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a reported cstate change: term changed from 0 to 1, leader changed from <none> to 45c2f2257c16439899c1f2fcf6ea235a (127.11.251.195). New cstate: current_term: 1 leader_uuid: "45c2f2257c16439899c1f2fcf6ea235a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c2f2257c16439899c1f2fcf6ea235a" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 42901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8628587183e141879420c17baaca6bcd" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 45607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50a58b48e19943c5a8bdcefe69d60022" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46845 } health_report { overall_health: UNKNOWN } } }
W20260927 18:42:58.535852 29748 tablet.cc:2369] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:42:58.549942 29760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:58.547044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40356 (local address 127.11.251.195:42901)
0927 18:42:58.547126 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:58.547128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:58.547146 (+ 18us) server_negotiation.cc:407] Connection header received
0927 18:42:58.547193 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:58.547194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:58.547216 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:58.547289 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:42:58.547508 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.548106 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.548925 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.549159 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.549229 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:58.549395 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:42:58.549533 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:42:58.549669 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:58.549731 (+ 62us) server_negotiation.cc:299] Negotiation successful
0927 18:42:58.549756 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 18:42:58.552543 29567 raft_consensus.cc:1261] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd [term 1 FOLLOWER]: Refusing update from remote peer 45c2f2257c16439899c1f2fcf6ea235a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:42:58.552533 29433 raft_consensus.cc:1261] T 5b7f1af070bc43b7b322d459f16a8a6e P 50a58b48e19943c5a8bdcefe69d60022 [term 1 FOLLOWER]: Refusing update from remote peer 45c2f2257c16439899c1f2fcf6ea235a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:42:58.553048 29772 consensus_queue.cc:1035] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8628587183e141879420c17baaca6bcd" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 45607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:42:58.553259 29774 consensus_queue.cc:1035] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a [LEADER]: Connected to new peer: Peer: permanent_uuid: "50a58b48e19943c5a8bdcefe69d60022" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 46845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:42:58.556903 29778 mvcc.cc:205] Tried to move back new op lower bound from 7334029633747066880 to 7334029633648676864. Current Snapshot: MvccSnapshot[applied={T|T < 7334029633747066880}]
I20260927 18:42:58.557792 29779 mvcc.cc:205] Tried to move back new op lower bound from 7334029633747066880 to 7334029633648676864. Current Snapshot: MvccSnapshot[applied={T|T < 7334029633747066880}]
I20260927 18:42:58.559484 29288 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:42:58.559612 29288 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:42:58.560613 29288 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1587608a19a44bcdbd1d282a23274030: Sending DeleteTablet for 3 replicas of tablet 5b7f1af070bc43b7b322d459f16a8a6e
I20260927 18:42:58.561067 29681 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b7f1af070bc43b7b322d459f16a8a6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:42:58 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:40346
I20260927 18:42:58.561333 29413 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b7f1af070bc43b7b322d459f16a8a6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:42:58 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:52130
I20260927 18:42:58.561650 29788 tablet_replica.cc:330] T 5b7f1af070bc43b7b322d459f16a8a6e P 50a58b48e19943c5a8bdcefe69d60022: stopping tablet replica
I20260927 18:42:58.561944 29787 tablet_replica.cc:330] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a: stopping tablet replica
I20260927 18:42:58.562026 29547 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b7f1af070bc43b7b322d459f16a8a6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:42:58 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:43962
I20260927 18:42:58.562106 29787 raft_consensus.cc:2229] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:42:58.562135 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 29350
I20260927 18:42:58.562247 29787 raft_consensus.cc:2258] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:42:58.563789 29787 ts_tablet_manager.cc:1905] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:42:58.564388 29789 tablet_replica.cc:330] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd: stopping tablet replica
I20260927 18:42:58.564613 29789 raft_consensus.cc:2229] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:42:58.564810 29789 raft_consensus.cc:2258] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:42:58.565443 29789 ts_tablet_manager.cc:1905] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:42:58.566128 29787 ts_tablet_manager.cc:1918] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:42:58.566288 29787 log.cc:1196] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/5b7f1af070bc43b7b322d459f16a8a6e
I20260927 18:42:58.566766 29787 ts_tablet_manager.cc:1939] T 5b7f1af070bc43b7b322d459f16a8a6e P 45c2f2257c16439899c1f2fcf6ea235a: Deleting consensus metadata
I20260927 18:42:58.567813 29789 ts_tablet_manager.cc:1918] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:42:58.567914 29789 log.cc:1196] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/5b7f1af070bc43b7b322d459f16a8a6e
I20260927 18:42:58.568190 29789 ts_tablet_manager.cc:1939] T 5b7f1af070bc43b7b322d459f16a8a6e P 8628587183e141879420c17baaca6bcd: Deleting consensus metadata
I20260927 18:42:58.569054 29275 catalog_manager.cc:4841] TS 8628587183e141879420c17baaca6bcd (127.11.251.194:45607): tablet 5b7f1af070bc43b7b322d459f16a8a6e (table test-table [id=aff37b09c67646dd8ffb2dfbbc39078c]) successfully deleted
I20260927 18:42:58.569849 29273 catalog_manager.cc:4841] TS 45c2f2257c16439899c1f2fcf6ea235a (127.11.251.195:42901): tablet 5b7f1af070bc43b7b322d459f16a8a6e (table test-table [id=aff37b09c67646dd8ffb2dfbbc39078c]) successfully deleted
W20260927 18:42:58.570103 29273 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.251.193:46845 (error 108)
I20260927 18:42:58.570349 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 29484
I20260927 18:42:58.571159 29759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:58.570619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:46845 (local address 127.0.0.1:52132)
0927 18:42:58.571019 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:58.571109 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.193:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362}
W20260927 18:42:58.571316 29273 catalog_manager.cc:4584] TS 50a58b48e19943c5a8bdcefe69d60022 (127.11.251.193:46845): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5b7f1af070bc43b7b322d459f16a8a6e: Network error: Client connection negotiation failed: client connection to 127.11.251.193:46845: connect: Connection refused (error 111)
I20260927 18:42:58.577688 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 29618
I20260927 18:42:58.583842 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 29256
2026-09-27T18:42:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2235 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-27T18:42:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:42:58Z Disabled control of system clock
I20260927 18:42:58.617130 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:37455
--webserver_interface=127.11.251.254
--webserver_port=0
--builtin_ntp_servers=127.11.251.212:37159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:37455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:42:58.710860 29794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:58.711187 29794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:58.711213 29794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:58.712652 29794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:42:58.712707 29794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:58.712721 29794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:58.712735 29794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:42:58.712747 29794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:42:58.714372 29794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:37159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:37455
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:37455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:42:58.714658 29794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:58.714991 29794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:58.720811 29802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:58.720988 29800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:58.721066 29794 server_base.cc:1034] running on GCE node
W20260927 18:42:58.721103 29799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:58.721884 29794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:58.722281 29794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:58.723491 29794 hybrid_clock.cc:648] HybridClock initialized: now 1790534578723449 us; error 74 us; skew 500 ppm
I20260927 18:42:58.724980 29794 webserver.cc:460] Webserver started at http://127.11.251.254:39269/ using document root <none> and password file <none>
I20260927 18:42:58.725301 29794 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:58.725381 29794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:58.725549 29794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:42:58.726768 29794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "47fd517984e64d84bb04d7fe56d43b00"
format_stamp: "Formatted at 2026-09-27 18:42:58 on dist-test-slave-z47j"
server_key: "7f29ce9ec5600fce33eb5806f7878d56"
server_key_iv: "81b97800747b960df83f42fd7142669a"
server_key_version: "encryptionkey@0"
I20260927 18:42:58.727150 29794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "47fd517984e64d84bb04d7fe56d43b00"
format_stamp: "Formatted at 2026-09-27 18:42:58 on dist-test-slave-z47j"
server_key: "7f29ce9ec5600fce33eb5806f7878d56"
server_key_iv: "81b97800747b960df83f42fd7142669a"
server_key_version: "encryptionkey@0"
I20260927 18:42:58.728655 29794 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:42:58.729642 29808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:58.729985 29794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:42:58.730146 29794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "47fd517984e64d84bb04d7fe56d43b00"
format_stamp: "Formatted at 2026-09-27 18:42:58 on dist-test-slave-z47j"
server_key: "7f29ce9ec5600fce33eb5806f7878d56"
server_key_iv: "81b97800747b960df83f42fd7142669a"
server_key_version: "encryptionkey@0"
I20260927 18:42:58.730244 29794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:58.740899 29794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:58.741264 29794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:58.741433 29794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:58.745982 29794 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:37455
I20260927 18:42:58.746096 29860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:37455 every 8 connection(s)
I20260927 18:42:58.746444 29794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:42:58.747697 29861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:42:58.751006 29861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00: Bootstrap starting.
I20260927 18:42:58.751754 29861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00: Neither blocks nor log segments found. Creating new log.
I20260927 18:42:58.752121 29861 log.cc:824] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:58.752995 29861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00: No bootstrap required, opened a new log
I20260927 18:42:58.753368 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 29794
I20260927 18:42:58.753523 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:42:58.753818 12271 external_mini_cluster.cc:1428] Setting key 5503e4b4ef4a25e419c1722cddada77c
I20260927 18:42:58.755337 29861 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47fd517984e64d84bb04d7fe56d43b00" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37455 } }
I20260927 18:42:58.755543 29861 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:42:58.755585 29861 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47fd517984e64d84bb04d7fe56d43b00, State: Initialized, Role: FOLLOWER
I20260927 18:42:58.755775 29861 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47fd517984e64d84bb04d7fe56d43b00" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37455 } }
I20260927 18:42:58.755882 29861 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:42:58.755921 29861 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:42:58.755992 29861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:42:58.756649 29861 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47fd517984e64d84bb04d7fe56d43b00" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37455 } }
I20260927 18:42:58.756783 29861 leader_election.cc:302] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47fd517984e64d84bb04d7fe56d43b00; no voters:
I20260927 18:42:58.757031 29861 leader_election.cc:288] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:42:58.757076 29866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:42:58.757198 29866 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00 [term 1 LEADER]: Becoming Leader. State: Replica: 47fd517984e64d84bb04d7fe56d43b00, State: Running, Role: LEADER
I20260927 18:42:58.757375 29861 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:42:58.757359 29866 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47fd517984e64d84bb04d7fe56d43b00" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37455 } }
I20260927 18:42:58.758023 29868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47fd517984e64d84bb04d7fe56d43b00. Latest consensus state: current_term: 1 leader_uuid: "47fd517984e64d84bb04d7fe56d43b00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47fd517984e64d84bb04d7fe56d43b00" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37455 } } }
I20260927 18:42:58.757994 29867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47fd517984e64d84bb04d7fe56d43b00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47fd517984e64d84bb04d7fe56d43b00" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37455 } } }
I20260927 18:42:58.758188 29868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00 [sys.catalog]: This master's current role is: LEADER
I20260927 18:42:58.758206 29867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00 [sys.catalog]: This master's current role is: LEADER
I20260927 18:42:58.758659 29875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:42:58.759996 29875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:42:58.761790 29865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:58.754407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51588 (local address 127.11.251.254:37455)
0927 18:42:58.754674 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:58.754684 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:58.754718 (+ 34us) server_negotiation.cc:407] Connection header received
0927 18:42:58.755348 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:58.755357 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:58.755430 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:58.755615 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:58.755813 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.757332 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.758353 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.759163 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.759383 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:58.760916 (+ 1533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:58.760921 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:58.760921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:42:58.761109 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:58.761180 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:42:58.761197 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:58.761271 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:58.761409 (+ 138us) server_negotiation.cc:299] Negotiation successful
0927 18:42:58.761511 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":97,"threads_started":1}
I20260927 18:42:58.762210 29875 catalog_manager.cc:1347] Generated new cluster ID: 6331c103ebb444c1a2652b4da0138606
I20260927 18:42:58.762274 29875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:42:58.781900 29875 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:42:58.782488 29875 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:42:58.792095 29875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00: Generated new TSK 0
I20260927 18:42:58.792303 29875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:42:58.796350 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:37455
--builtin_ntp_servers=127.11.251.212:37159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:42:58.883438 29885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:58.883656 29885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:58.883677 29885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:58.884824 29885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:58.884867 29885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:58.884893 29885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:42:58.886425 29885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:37159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:37455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:42:58.886678 29885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:58.887017 29885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:58.892520 29890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:58.892536 29893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:58.892549 29891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:58.893044 29885 server_base.cc:1034] running on GCE node
I20260927 18:42:58.893266 29885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:58.893550 29885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:58.894764 29885 hybrid_clock.cc:648] HybridClock initialized: now 1790534578894730 us; error 55 us; skew 500 ppm
I20260927 18:42:58.896297 29885 webserver.cc:460] Webserver started at http://127.11.251.193:35453/ using document root <none> and password file <none>
I20260927 18:42:58.896592 29885 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:58.896672 29885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:58.897086 29885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:42:58.898377 29885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "e231de6663c247da9546be5c98efb4d6"
format_stamp: "Formatted at 2026-09-27 18:42:58 on dist-test-slave-z47j"
server_key: "75b510119d045f6fe69607c9098d5c14"
server_key_iv: "7a13a7a97bb0102e856ffdd4456e92f4"
server_key_version: "encryptionkey@0"
I20260927 18:42:58.898756 29885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "e231de6663c247da9546be5c98efb4d6"
format_stamp: "Formatted at 2026-09-27 18:42:58 on dist-test-slave-z47j"
server_key: "75b510119d045f6fe69607c9098d5c14"
server_key_iv: "7a13a7a97bb0102e856ffdd4456e92f4"
server_key_version: "encryptionkey@0"
I20260927 18:42:58.900341 29885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:42:58.901310 29899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:58.901789 29885 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:42:58.901903 29885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "e231de6663c247da9546be5c98efb4d6"
format_stamp: "Formatted at 2026-09-27 18:42:58 on dist-test-slave-z47j"
server_key: "75b510119d045f6fe69607c9098d5c14"
server_key_iv: "7a13a7a97bb0102e856ffdd4456e92f4"
server_key_version: "encryptionkey@0"
I20260927 18:42:58.902006 29885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:58.922781 29885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:58.923130 29885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:58.923300 29885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:58.923555 29885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:42:58.923913 29885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:42:58.923974 29885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:58.924019 29885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:42:58.924044 29885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:58.930760 29885 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:39321
I20260927 18:42:58.930852 30012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:39321 every 8 connection(s)
I20260927 18:42:58.931267 29885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:42:58.931561 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 29885
I20260927 18:42:58.931660 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:42:58.931917 12271 external_mini_cluster.cc:1428] Setting key 5f9f3a3bb72e7545ccbc2de323a7763e
I20260927 18:42:58.933318 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:37455
--builtin_ntp_servers=127.11.251.212:37159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 18:42:58.938789 29865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:58.932325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:56423 (local address 127.11.251.254:37455)
0927 18:42:58.932572 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:58.932575 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:58.932753 (+ 178us) server_negotiation.cc:407] Connection header received
0927 18:42:58.933706 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:58.933710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:58.933731 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:58.933791 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:58.934866 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.935742 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.936879 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.937140 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.937246 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:58.938454 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:58.938456 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:58.938456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:42:58.938462 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:58.938477 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:42:58.938481 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:58.938540 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:58.938655 (+ 115us) server_negotiation.cc:299] Negotiation successful
0927 18:42:58.938694 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207}
I20260927 18:42:58.939073 30015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:58.932252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:37455 (local address 127.11.251.193:56423)
0927 18:42:58.932533 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:58.932575 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:58.933558 (+ 983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:58.933892 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:58.933897 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:58.933909 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:58.934643 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:58.934651 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.935883 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.935884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:58.936768 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:58.936770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:58.937260 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:58.937262 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:58.937458 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:58.938219 (+ 761us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:42:58.938220 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:58.938255 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:42:58.938284 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:58.938555 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:58.938565 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:58.938573 (+ 8us) client_negotiation.cc:769] Sending connection context
0927 18:42:58.938659 (+ 86us) client_negotiation.cc:240] Negotiation successful
0927 18:42:58.938793 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":135,"threads_started":1}
I20260927 18:42:58.939769 30013 heartbeater.cc:347] Connected to a master server at 127.11.251.254:37455
I20260927 18:42:58.939929 30013 heartbeater.cc:464] Registering TS with master...
I20260927 18:42:58.940163 30013 heartbeater.cc:511] Master 127.11.251.254:37455 requested a full tablet report, sending...
I20260927 18:42:58.940886 29825 ts_manager.cc:194] Registered new tserver with Master: e231de6663c247da9546be5c98efb4d6 (127.11.251.193:39321)
I20260927 18:42:58.941992 29825 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.251.193:56423
W20260927 18:42:59.019330 30016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:59.019542 30016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:59.019564 30016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:59.020742 30016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:59.020790 30016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:59.020816 30016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:42:59.022111 30016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:37159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:37455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:42:59.022307 30016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:59.022549 30016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:59.028052 30021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:59.028112 30016 server_base.cc:1034] running on GCE node
W20260927 18:42:59.028052 30024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:59.028067 30022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:59.028518 30016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:59.028784 30016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:59.029963 30016 hybrid_clock.cc:648] HybridClock initialized: now 1790534579029924 us; error 65 us; skew 500 ppm
I20260927 18:42:59.031358 30016 webserver.cc:460] Webserver started at http://127.11.251.194:46021/ using document root <none> and password file <none>
I20260927 18:42:59.031641 30016 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:59.031697 30016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:59.031888 30016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:42:59.032980 30016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "7a5dae59ce90498f99743622eb7ef2db"
format_stamp: "Formatted at 2026-09-27 18:42:59 on dist-test-slave-z47j"
server_key: "59a233b5d7be34866dd06c83db87343c"
server_key_iv: "60b5e73da6945a8ccd5ff43df2123a1b"
server_key_version: "encryptionkey@0"
I20260927 18:42:59.033357 30016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "7a5dae59ce90498f99743622eb7ef2db"
format_stamp: "Formatted at 2026-09-27 18:42:59 on dist-test-slave-z47j"
server_key: "59a233b5d7be34866dd06c83db87343c"
server_key_iv: "60b5e73da6945a8ccd5ff43df2123a1b"
server_key_version: "encryptionkey@0"
I20260927 18:42:59.034932 30016 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:42:59.036012 30030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:59.036304 30016 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:42:59.036422 30016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "7a5dae59ce90498f99743622eb7ef2db"
format_stamp: "Formatted at 2026-09-27 18:42:59 on dist-test-slave-z47j"
server_key: "59a233b5d7be34866dd06c83db87343c"
server_key_iv: "60b5e73da6945a8ccd5ff43df2123a1b"
server_key_version: "encryptionkey@0"
I20260927 18:42:59.036515 30016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:59.045928 30016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:59.046245 30016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:59.046402 30016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:59.046645 30016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:42:59.046968 30016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:42:59.047029 30016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:59.047066 30016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:42:59.047113 30016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:59.054122 30016 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:36529
I20260927 18:42:59.054171 30143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:36529 every 8 connection(s)
I20260927 18:42:59.054495 30016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:42:59.060420 29865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:59.055254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:35245 (local address 127.11.251.254:37455)
0927 18:42:59.055324 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:59.055326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:59.055571 (+ 245us) server_negotiation.cc:407] Connection header received
0927 18:42:59.056092 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:59.056093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:59.056106 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:59.056200 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:59.056948 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.057659 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.058864 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.059123 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.059192 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:59.060072 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:59.060073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:59.060074 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:42:59.060079 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:59.060094 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:42:59.060099 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:59.060147 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:59.060260 (+ 113us) server_negotiation.cc:299] Negotiation successful
0927 18:42:59.060288 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 18:42:59.060492 30146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:59.055190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:37455 (local address 127.11.251.194:35245)
0927 18:42:59.055399 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:59.055443 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:59.055980 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:59.056317 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:59.056320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:59.056329 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:59.056831 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:59.056834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.057781 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.057782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:59.058752 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:59.058755 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.059215 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.059216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:59.059338 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:59.059929 (+ 591us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:42:59.059929 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:59.059958 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:42:59.059979 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:59.060169 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:59.060173 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:59.060175 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:42:59.060234 (+ 59us) client_negotiation.cc:240] Negotiation successful
0927 18:42:59.060331 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":118,"threads_started":1}
I20260927 18:42:59.060644 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 30016
I20260927 18:42:59.060763 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:42:59.061010 12271 external_mini_cluster.cc:1428] Setting key 7388199ffd941eac47fa46a9f1ad1e16
I20260927 18:42:59.061051 30144 heartbeater.cc:347] Connected to a master server at 127.11.251.254:37455
I20260927 18:42:59.061183 30144 heartbeater.cc:464] Registering TS with master...
I20260927 18:42:59.061415 30144 heartbeater.cc:511] Master 127.11.251.254:37455 requested a full tablet report, sending...
I20260927 18:42:59.061899 29825 ts_manager.cc:194] Registered new tserver with Master: 7a5dae59ce90498f99743622eb7ef2db (127.11.251.194:36529)
I20260927 18:42:59.062302 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:37455
--builtin_ntp_servers=127.11.251.212:37159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 18:42:59.062310 29825 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.251.194:35245
W20260927 18:42:59.156654 30147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:59.156930 30147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:59.156957 30147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:59.158717 30147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:59.158807 30147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:59.158876 30147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:42:59.160537 30147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:37159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:37455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:42:59.160862 30147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:59.161252 30147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:59.167219 30155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:59.167315 30152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:59.167273 30153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:59.167475 30147 server_base.cc:1034] running on GCE node
I20260927 18:42:59.167851 30147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:59.168192 30147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:59.169384 30147 hybrid_clock.cc:648] HybridClock initialized: now 1790534579169360 us; error 62 us; skew 500 ppm
I20260927 18:42:59.170979 30147 webserver.cc:460] Webserver started at http://127.11.251.195:34133/ using document root <none> and password file <none>
I20260927 18:42:59.171284 30147 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:59.171362 30147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:59.171514 30147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:42:59.172741 30147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "89fdb9a1f1af4e7db626953c9b2bf52c"
format_stamp: "Formatted at 2026-09-27 18:42:59 on dist-test-slave-z47j"
server_key: "6186ac3c7e4a6afc4ac073415cdc2686"
server_key_iv: "dff2724999ba8ec0c404da44fe5f9b17"
server_key_version: "encryptionkey@0"
I20260927 18:42:59.173132 30147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "89fdb9a1f1af4e7db626953c9b2bf52c"
format_stamp: "Formatted at 2026-09-27 18:42:59 on dist-test-slave-z47j"
server_key: "6186ac3c7e4a6afc4ac073415cdc2686"
server_key_iv: "dff2724999ba8ec0c404da44fe5f9b17"
server_key_version: "encryptionkey@0"
I20260927 18:42:59.174749 30147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:42:59.175629 30161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:59.176012 30147 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:42:59.176152 30147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "89fdb9a1f1af4e7db626953c9b2bf52c"
format_stamp: "Formatted at 2026-09-27 18:42:59 on dist-test-slave-z47j"
server_key: "6186ac3c7e4a6afc4ac073415cdc2686"
server_key_iv: "dff2724999ba8ec0c404da44fe5f9b17"
server_key_version: "encryptionkey@0"
I20260927 18:42:59.176252 30147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:59.192678 30147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:59.193029 30147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:59.193200 30147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:59.193470 30147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:42:59.193878 30147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:42:59.193939 30147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:59.193984 30147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:42:59.194011 30147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:59.201320 30147 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:44217
I20260927 18:42:59.201485 30274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:44217 every 8 connection(s)
I20260927 18:42:59.201810 30147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:42:59.208007 29865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:59.202779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:43051 (local address 127.11.251.254:37455)
0927 18:42:59.202864 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:59.202867 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:59.203054 (+ 187us) server_negotiation.cc:407] Connection header received
0927 18:42:59.203601 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:59.203603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:59.203615 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:59.203704 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:59.204424 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.205042 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.206515 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.206720 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.206787 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:59.207739 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:59.207740 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:59.207741 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:42:59.207746 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:59.207763 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:42:59.207767 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:59.207816 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:59.207893 (+ 77us) server_negotiation.cc:299] Negotiation successful
0927 18:42:59.207929 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 18:42:59.208276 30277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:59.202738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:37455 (local address 127.11.251.195:43051)
0927 18:42:59.202926 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:59.202968 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:59.203472 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:59.203762 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:59.203765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:59.203774 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:59.204311 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:59.204316 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.205166 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.205167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:59.206412 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:59.206415 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.206810 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.206811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:59.206923 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:59.207590 (+ 667us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:42:59.207590 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:59.207623 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:42:59.207648 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:59.207818 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:59.207821 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:59.207823 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:42:59.207877 (+ 54us) client_negotiation.cc:240] Negotiation successful
0927 18:42:59.207956 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260927 18:42:59.208925 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 30147
I20260927 18:42:59.209065 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:42:59.209283 30275 heartbeater.cc:347] Connected to a master server at 127.11.251.254:37455
I20260927 18:42:59.209334 12271 external_mini_cluster.cc:1428] Setting key 4bac8616546040d660ea596b76f60cac
I20260927 18:42:59.209393 30275 heartbeater.cc:464] Registering TS with master...
I20260927 18:42:59.209674 30275 heartbeater.cc:511] Master 127.11.251.254:37455 requested a full tablet report, sending...
I20260927 18:42:59.210253 29825 ts_manager.cc:194] Registered new tserver with Master: 89fdb9a1f1af4e7db626953c9b2bf52c (127.11.251.195:44217)
I20260927 18:42:59.210834 29825 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.251.195:43051
I20260927 18:42:59.211001 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:42:59.216507 29865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:59.212870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51590 (local address 127.11.251.254:37455)
0927 18:42:59.212999 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:59.213001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:59.213130 (+ 129us) server_negotiation.cc:407] Connection header received
0927 18:42:59.213215 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:59.213216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:59.213232 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:59.213277 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:59.213490 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.214115 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.215148 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.215363 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.215444 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:59.216191 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:59.216192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:59.216193 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:42:59.216198 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:59.216219 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:42:59.216223 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:59.216290 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:59.216354 (+ 64us) server_negotiation.cc:299] Negotiation successful
0927 18:42:59.216380 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260927 18:42:59.218611 29825 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51590:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:42:59.220924 29825 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:42:59.228544 30286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:59.223878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:44217 (local address 127.0.0.1:45642)
0927 18:42:59.224267 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:59.224283 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:59.224349 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:59.224743 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:59.224744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:59.224747 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:42:59.224810 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:59.224813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.225919 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.225921 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:59.227425 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:59.227428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.228188 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.228189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:59.228266 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:59.228281 (+ 15us) client_negotiation.cc:769] Sending connection context
0927 18:42:59.228359 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 18:42:59.228440 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":137,"thread_start_us":189,"threads_started":1}
I20260927 18:42:59.228605 30287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:59.224194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45642 (local address 127.11.251.195:44217)
0927 18:42:59.224521 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:59.224527 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:59.224538 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:42:59.224589 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:59.224591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:59.224626 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:59.224885 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:42:59.224933 (+ 48us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.225831 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.227535 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.228081 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.228179 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:59.228281 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:59.228373 (+ 92us) server_negotiation.cc:299] Negotiation successful
0927 18:42:59.228414 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":110,"threads_started":1}
I20260927 18:42:59.228678 30285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:59.223767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:39321 (local address 127.0.0.1:49062)
0927 18:42:59.224164 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:59.224208 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:59.224279 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:59.224672 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:59.224676 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:59.224678 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:42:59.224777 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:59.224780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.226147 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.226148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:59.227319 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:59.227321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.228363 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.228365 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:59.228457 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:59.228468 (+ 11us) client_negotiation.cc:769] Sending connection context
0927 18:42:59.228521 (+ 53us) client_negotiation.cc:240] Negotiation successful
0927 18:42:59.228580 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":72,"thread_start_us":99,"threads_started":1}
I20260927 18:42:59.228929 30284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:59.223922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49062 (local address 127.11.251.193:39321)
0927 18:42:59.224430 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:59.224435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:59.224445 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:42:59.224488 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:59.224490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:59.224507 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:59.224576 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:42:59.224860 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.226049 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.227412 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.228152 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.228232 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:59.228322 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:59.228750 (+ 428us) server_negotiation.cc:299] Negotiation successful
0927 18:42:59.228853 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":107,"threads_started":1}
I20260927 18:42:59.228974 30288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:59.224300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38684 (local address 127.11.251.194:36529)
0927 18:42:59.224521 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:59.224527 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:59.224789 (+ 262us) server_negotiation.cc:407] Connection header received
0927 18:42:59.224876 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:59.224879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:59.224896 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:59.224981 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:42:59.225119 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.226184 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.227510 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.228171 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.228232 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:59.228322 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:59.228629 (+ 307us) server_negotiation.cc:299] Negotiation successful
0927 18:42:59.228889 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":110,"threads_started":1}
I20260927 18:42:59.229136 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:59.224034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:36529 (local address 127.0.0.1:38684)
0927 18:42:59.224663 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:59.224673 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:59.224723 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:59.225004 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:59.225005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:59.225007 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:42:59.225045 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:59.225046 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.226276 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.226277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:59.227428 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:59.227430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.228366 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.228367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:59.228457 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:59.228469 (+ 12us) client_negotiation.cc:769] Sending connection context
0927 18:42:59.228521 (+ 52us) client_negotiation.cc:240] Negotiation successful
0927 18:42:59.228580 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":7,"thread_start_us":118,"threads_started":1}
I20260927 18:42:59.229692 30209 tablet_service.cc:1467] Processing CreateTablet for tablet bb8e06ab18614c9e91b545dea610f00e (DEFAULT_TABLE table=test-table [id=61181176876c440ba3c7e2b05bc9f547]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:42:59.230140 30209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb8e06ab18614c9e91b545dea610f00e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:42:59.230039 29947 tablet_service.cc:1467] Processing CreateTablet for tablet bb8e06ab18614c9e91b545dea610f00e (DEFAULT_TABLE table=test-table [id=61181176876c440ba3c7e2b05bc9f547]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:42:59.230461 29947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb8e06ab18614c9e91b545dea610f00e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:42:59.233282 30289 tablet_bootstrap.cc:492] T bb8e06ab18614c9e91b545dea610f00e P e231de6663c247da9546be5c98efb4d6: Bootstrap starting.
I20260927 18:42:59.233937 30290 tablet_bootstrap.cc:492] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c: Bootstrap starting.
I20260927 18:42:59.234241 30289 tablet_bootstrap.cc:654] T bb8e06ab18614c9e91b545dea610f00e P e231de6663c247da9546be5c98efb4d6: Neither blocks nor log segments found. Creating new log.
I20260927 18:42:59.234087 30078 tablet_service.cc:1467] Processing CreateTablet for tablet bb8e06ab18614c9e91b545dea610f00e (DEFAULT_TABLE table=test-table [id=61181176876c440ba3c7e2b05bc9f547]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:42:59.234524 30078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb8e06ab18614c9e91b545dea610f00e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:42:59.234630 30289 log.cc:824] T bb8e06ab18614c9e91b545dea610f00e P e231de6663c247da9546be5c98efb4d6: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:59.234747 30290 tablet_bootstrap.cc:654] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c: Neither blocks nor log segments found. Creating new log.
I20260927 18:42:59.235077 30290 log.cc:824] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:59.235638 30289 tablet_bootstrap.cc:492] T bb8e06ab18614c9e91b545dea610f00e P e231de6663c247da9546be5c98efb4d6: No bootstrap required, opened a new log
I20260927 18:42:59.235762 30289 ts_tablet_manager.cc:1397] T bb8e06ab18614c9e91b545dea610f00e P e231de6663c247da9546be5c98efb4d6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:42:59.235877 30290 tablet_bootstrap.cc:492] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c: No bootstrap required, opened a new log
I20260927 18:42:59.235978 30290 ts_tablet_manager.cc:1397] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:42:59.238564 30293 tablet_bootstrap.cc:492] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db: Bootstrap starting.
I20260927 18:42:59.238329 30290 raft_consensus.cc:348] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e231de6663c247da9546be5c98efb4d6" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39321 } } peers { permanent_uuid: "89fdb9a1f1af4e7db626953c9b2bf52c" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44217 } } peers { permanent_uuid: "7a5dae59ce90498f99743622eb7ef2db" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36529 } }
I20260927 18:42:59.238758 30290 raft_consensus.cc:374] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:42:59.238809 30290 raft_consensus.cc:729] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89fdb9a1f1af4e7db626953c9b2bf52c, State: Initialized, Role: FOLLOWER
I20260927 18:42:59.238965 30290 consensus_queue.cc:260] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e231de6663c247da9546be5c98efb4d6" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39321 } } peers { permanent_uuid: "89fdb9a1f1af4e7db626953c9b2bf52c" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44217 } } peers { permanent_uuid: "7a5dae59ce90498f99743622eb7ef2db" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36529 } }
I20260927 18:42:59.238888 30289 raft_consensus.cc:348] T bb8e06ab18614c9e91b545dea610f00e P e231de6663c247da9546be5c98efb4d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e231de6663c247da9546be5c98efb4d6" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39321 } } peers { permanent_uuid: "89fdb9a1f1af4e7db626953c9b2bf52c" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44217 } } peers { permanent_uuid: "7a5dae59ce90498f99743622eb7ef2db" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36529 } }
I20260927 18:42:59.239094 30289 raft_consensus.cc:374] T bb8e06ab18614c9e91b545dea610f00e P e231de6663c247da9546be5c98efb4d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:42:59.239140 30289 raft_consensus.cc:729] T bb8e06ab18614c9e91b545dea610f00e P e231de6663c247da9546be5c98efb4d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e231de6663c247da9546be5c98efb4d6, State: Initialized, Role: FOLLOWER
I20260927 18:42:59.239266 30290 ts_tablet_manager.cc:1428] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:42:59.239253 30289 consensus_queue.cc:260] T bb8e06ab18614c9e91b545dea610f00e P e231de6663c247da9546be5c98efb4d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e231de6663c247da9546be5c98efb4d6" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39321 } } peers { permanent_uuid: "89fdb9a1f1af4e7db626953c9b2bf52c" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44217 } } peers { permanent_uuid: "7a5dae59ce90498f99743622eb7ef2db" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36529 } }
I20260927 18:42:59.239362 30293 tablet_bootstrap.cc:654] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db: Neither blocks nor log segments found. Creating new log.
I20260927 18:42:59.239409 30275 heartbeater.cc:503] Master 127.11.251.254:37455 was elected leader, sending a full tablet report...
I20260927 18:42:59.239532 30289 ts_tablet_manager.cc:1428] T bb8e06ab18614c9e91b545dea610f00e P e231de6663c247da9546be5c98efb4d6: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260927 18:42:59.239602 30013 heartbeater.cc:503] Master 127.11.251.254:37455 was elected leader, sending a full tablet report...
I20260927 18:42:59.239769 30293 log.cc:824] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:59.241005 30293 tablet_bootstrap.cc:492] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db: No bootstrap required, opened a new log
I20260927 18:42:59.241104 30293 ts_tablet_manager.cc:1397] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 18:42:59.243232 30293 raft_consensus.cc:348] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e231de6663c247da9546be5c98efb4d6" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39321 } } peers { permanent_uuid: "89fdb9a1f1af4e7db626953c9b2bf52c" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44217 } } peers { permanent_uuid: "7a5dae59ce90498f99743622eb7ef2db" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36529 } }
I20260927 18:42:59.243395 30293 raft_consensus.cc:374] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:42:59.243435 30293 raft_consensus.cc:729] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a5dae59ce90498f99743622eb7ef2db, State: Initialized, Role: FOLLOWER
I20260927 18:42:59.243567 30293 consensus_queue.cc:260] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e231de6663c247da9546be5c98efb4d6" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39321 } } peers { permanent_uuid: "89fdb9a1f1af4e7db626953c9b2bf52c" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44217 } } peers { permanent_uuid: "7a5dae59ce90498f99743622eb7ef2db" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36529 } }
I20260927 18:42:59.243835 30293 ts_tablet_manager.cc:1428] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260927 18:42:59.243918 30144 heartbeater.cc:503] Master 127.11.251.254:37455 was elected leader, sending a full tablet report...
W20260927 18:42:59.305173 30145 tablet.cc:2369] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:42:59.432327 30014 tablet.cc:2369] T bb8e06ab18614c9e91b545dea610f00e P e231de6663c247da9546be5c98efb4d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:42:59.452514 30276 tablet.cc:2369] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:42:59.529676 30297 raft_consensus.cc:482] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:42:59.529873 30297 raft_consensus.cc:504] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e231de6663c247da9546be5c98efb4d6" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39321 } } peers { permanent_uuid: "89fdb9a1f1af4e7db626953c9b2bf52c" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44217 } } peers { permanent_uuid: "7a5dae59ce90498f99743622eb7ef2db" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36529 } }
I20260927 18:42:59.530617 30297 leader_election.cc:288] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e231de6663c247da9546be5c98efb4d6 (127.11.251.193:39321), 89fdb9a1f1af4e7db626953c9b2bf52c (127.11.251.195:44217)
I20260927 18:42:59.534934 30146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:59.530665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:39321 (local address 127.11.251.194:45069)
0927 18:42:59.531755 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:59.531771 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:59.531867 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:59.532068 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:59.532069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:59.532071 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:42:59.532140 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:59.532142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.532854 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.532855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:59.534008 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:59.534009 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.534623 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.534624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:59.534716 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:59.534722 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:42:59.534805 (+ 83us) client_negotiation.cc:240] Negotiation successful
0927 18:42:59.534857 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1037,"mutex_wait_us":10,"thread_start_us":178,"threads_started":1}
I20260927 18:42:59.534931 30284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:59.531028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:45069 (local address 127.11.251.193:39321)
0927 18:42:59.531111 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:59.531113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:59.531849 (+ 736us) server_negotiation.cc:407] Connection header received
0927 18:42:59.531939 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:59.531940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:59.531954 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:59.532013 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:42:59.532208 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.532768 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.534098 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.534508 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.534623 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:59.534634 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:59.534812 (+ 178us) server_negotiation.cc:299] Negotiation successful
0927 18:42:59.534857 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 18:42:59.535063 30298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:59.530567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:44217 (local address 127.11.251.194:41205)
0927 18:42:59.530944 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:59.530960 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:59.531061 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:59.531302 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:59.531303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:59.531305 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:42:59.531435 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:59.531438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.532423 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.532423 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:59.533899 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:59.533903 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.534686 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.534687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:59.534780 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:59.534784 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:42:59.534903 (+ 119us) client_negotiation.cc:240] Negotiation successful
0927 18:42:59.534960 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318}
I20260927 18:42:59.535054 30287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:59.530674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:41205 (local address 127.11.251.195:44217)
0927 18:42:59.531000 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:59.531002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:59.531020 (+ 18us) server_negotiation.cc:407] Connection header received
0927 18:42:59.531170 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:59.531171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:59.531182 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:59.531236 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:42:59.531513 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.532312 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.534087 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.534548 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.534683 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:59.534694 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:59.534912 (+ 218us) server_negotiation.cc:299] Negotiation successful
0927 18:42:59.534958 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293}
I20260927 18:42:59.535425 29967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb8e06ab18614c9e91b545dea610f00e" candidate_uuid: "7a5dae59ce90498f99743622eb7ef2db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e231de6663c247da9546be5c98efb4d6" is_pre_election: true
I20260927 18:42:59.535610 29967 raft_consensus.cc:2454] T bb8e06ab18614c9e91b545dea610f00e P e231de6663c247da9546be5c98efb4d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a5dae59ce90498f99743622eb7ef2db in term 0.
I20260927 18:42:59.535569 30229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb8e06ab18614c9e91b545dea610f00e" candidate_uuid: "7a5dae59ce90498f99743622eb7ef2db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89fdb9a1f1af4e7db626953c9b2bf52c" is_pre_election: true
I20260927 18:42:59.535748 30229 raft_consensus.cc:2454] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a5dae59ce90498f99743622eb7ef2db in term 0.
I20260927 18:42:59.536005 30034 leader_election.cc:302] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a5dae59ce90498f99743622eb7ef2db, 89fdb9a1f1af4e7db626953c9b2bf52c; no voters:
I20260927 18:42:59.536146 30297 raft_consensus.cc:2781] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:42:59.536228 30297 raft_consensus.cc:482] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:42:59.536271 30297 raft_consensus.cc:3037] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:42:59.537074 30297 raft_consensus.cc:504] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e231de6663c247da9546be5c98efb4d6" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39321 } } peers { permanent_uuid: "89fdb9a1f1af4e7db626953c9b2bf52c" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44217 } } peers { permanent_uuid: "7a5dae59ce90498f99743622eb7ef2db" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36529 } }
I20260927 18:42:59.537238 30297 leader_election.cc:288] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db [CANDIDATE]: Term 1 election: Requested vote from peers e231de6663c247da9546be5c98efb4d6 (127.11.251.193:39321), 89fdb9a1f1af4e7db626953c9b2bf52c (127.11.251.195:44217)
I20260927 18:42:59.537395 30229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb8e06ab18614c9e91b545dea610f00e" candidate_uuid: "7a5dae59ce90498f99743622eb7ef2db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89fdb9a1f1af4e7db626953c9b2bf52c"
I20260927 18:42:59.537477 30229 raft_consensus.cc:3037] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:42:59.537443 29967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb8e06ab18614c9e91b545dea610f00e" candidate_uuid: "7a5dae59ce90498f99743622eb7ef2db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e231de6663c247da9546be5c98efb4d6"
I20260927 18:42:59.537662 29967 raft_consensus.cc:3037] T bb8e06ab18614c9e91b545dea610f00e P e231de6663c247da9546be5c98efb4d6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:42:59.538395 30229 raft_consensus.cc:2454] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a5dae59ce90498f99743622eb7ef2db in term 1.
I20260927 18:42:59.538578 30034 leader_election.cc:302] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a5dae59ce90498f99743622eb7ef2db, 89fdb9a1f1af4e7db626953c9b2bf52c; no voters:
I20260927 18:42:59.538691 30297 raft_consensus.cc:2781] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:42:59.538839 30297 raft_consensus.cc:686] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db [term 1 LEADER]: Becoming Leader. State: Replica: 7a5dae59ce90498f99743622eb7ef2db, State: Running, Role: LEADER
I20260927 18:42:59.538857 29967 raft_consensus.cc:2454] T bb8e06ab18614c9e91b545dea610f00e P e231de6663c247da9546be5c98efb4d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a5dae59ce90498f99743622eb7ef2db in term 1.
I20260927 18:42:59.539108 30297 consensus_queue.cc:237] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e231de6663c247da9546be5c98efb4d6" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39321 } } peers { permanent_uuid: "89fdb9a1f1af4e7db626953c9b2bf52c" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44217 } } peers { permanent_uuid: "7a5dae59ce90498f99743622eb7ef2db" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36529 } }
I20260927 18:42:59.540177 29825 catalog_manager.cc:5495] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a5dae59ce90498f99743622eb7ef2db (127.11.251.194). New cstate: current_term: 1 leader_uuid: "7a5dae59ce90498f99743622eb7ef2db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e231de6663c247da9546be5c98efb4d6" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89fdb9a1f1af4e7db626953c9b2bf52c" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a5dae59ce90498f99743622eb7ef2db" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 36529 } health_report { overall_health: HEALTHY } } }
I20260927 18:42:59.589614 30288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:59.586767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38694 (local address 127.11.251.194:36529)
0927 18:42:59.586875 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:59.586878 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:59.586894 (+ 16us) server_negotiation.cc:407] Connection header received
0927 18:42:59.586976 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:59.586977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:59.587032 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:59.587112 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:42:59.587305 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.587845 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.588682 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:59.588898 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:59.589013 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:59.589156 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:42:59.589299 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:42:59.589410 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:59.589427 (+ 17us) server_negotiation.cc:299] Negotiation successful
0927 18:42:59.589451 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260927 18:42:59.592078 30229 raft_consensus.cc:1261] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c [term 1 FOLLOWER]: Refusing update from remote peer 7a5dae59ce90498f99743622eb7ef2db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:42:59.592763 29967 raft_consensus.cc:1261] T bb8e06ab18614c9e91b545dea610f00e P e231de6663c247da9546be5c98efb4d6 [term 1 FOLLOWER]: Refusing update from remote peer 7a5dae59ce90498f99743622eb7ef2db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:42:59.592805 30297 consensus_queue.cc:1035] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db [LEADER]: Connected to new peer: Peer: permanent_uuid: "89fdb9a1f1af4e7db626953c9b2bf52c" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:42:59.593139 30297 consensus_queue.cc:1035] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db [LEADER]: Connected to new peer: Peer: permanent_uuid: "e231de6663c247da9546be5c98efb4d6" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 39321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:42:59.595681 30302 mvcc.cc:205] Tried to move back new op lower bound from 7334029638005370880 to 7334029637792784384. Current Snapshot: MvccSnapshot[applied={T|T < 7334029638005370880}]
I20260927 18:42:59.596506 30304 mvcc.cc:205] Tried to move back new op lower bound from 7334029638005370880 to 7334029637792784384. Current Snapshot: MvccSnapshot[applied={T|T < 7334029638005370880 or (T in {7334029638005370880})}]
I20260927 18:42:59.596781 30303 mvcc.cc:205] Tried to move back new op lower bound from 7334029638005370880 to 7334029637792784384. Current Snapshot: MvccSnapshot[applied={T|T < 7334029638005370880}]
I20260927 18:42:59.599969 29823 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:42:59.600101 29823 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:42:59.601212 29823 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47fd517984e64d84bb04d7fe56d43b00: Sending DeleteTablet for 3 replicas of tablet bb8e06ab18614c9e91b545dea610f00e
I20260927 18:42:59.601759 30078 tablet_service.cc:1514] Processing DeleteTablet for tablet bb8e06ab18614c9e91b545dea610f00e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:42:59 UTC) from {username='slave'} at 127.0.0.1:38684
I20260927 18:42:59.601788 29947 tablet_service.cc:1514] Processing DeleteTablet for tablet bb8e06ab18614c9e91b545dea610f00e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:42:59 UTC) from {username='slave'} at 127.0.0.1:49062
I20260927 18:42:59.602057 30209 tablet_service.cc:1514] Processing DeleteTablet for tablet bb8e06ab18614c9e91b545dea610f00e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:42:59 UTC) from {username='slave'} at 127.0.0.1:45642
I20260927 18:42:59.602349 30314 tablet_replica.cc:330] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c: stopping tablet replica
I20260927 18:42:59.602378 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 29885
I20260927 18:42:59.602546 30314 raft_consensus.cc:2229] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:42:59.602720 30314 raft_consensus.cc:2258] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:42:59.603611 30314 ts_tablet_manager.cc:1905] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:42:59.605135 30312 tablet_replica.cc:330] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db: stopping tablet replica
I20260927 18:42:59.605257 30312 raft_consensus.cc:2229] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:42:59.605420 30312 raft_consensus.cc:2258] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:42:59.605968 30312 ts_tablet_manager.cc:1905] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:42:59.606159 30314 ts_tablet_manager.cc:1918] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:42:59.606276 30314 log.cc:1196] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/bb8e06ab18614c9e91b545dea610f00e
I20260927 18:42:59.606606 30314 ts_tablet_manager.cc:1939] T bb8e06ab18614c9e91b545dea610f00e P 89fdb9a1f1af4e7db626953c9b2bf52c: Deleting consensus metadata
I20260927 18:42:59.607172 29812 catalog_manager.cc:4841] TS 89fdb9a1f1af4e7db626953c9b2bf52c (127.11.251.195:44217): tablet bb8e06ab18614c9e91b545dea610f00e (table test-table [id=61181176876c440ba3c7e2b05bc9f547]) successfully deleted
I20260927 18:42:59.607905 30312 ts_tablet_manager.cc:1918] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:42:59.607996 30312 log.cc:1196] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/bb8e06ab18614c9e91b545dea610f00e
I20260927 18:42:59.608237 30312 ts_tablet_manager.cc:1939] T bb8e06ab18614c9e91b545dea610f00e P 7a5dae59ce90498f99743622eb7ef2db: Deleting consensus metadata
I20260927 18:42:59.608946 29811 catalog_manager.cc:4841] TS 7a5dae59ce90498f99743622eb7ef2db (127.11.251.194:36529): tablet bb8e06ab18614c9e91b545dea610f00e (table test-table [id=61181176876c440ba3c7e2b05bc9f547]) successfully deleted
W20260927 18:42:59.610351 29811 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.251.193:39321 (error 108)
I20260927 18:42:59.610764 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 30016
I20260927 18:42:59.611034 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:59.610840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:39321 (local address 127.0.0.1:49070)
0927 18:42:59.610945 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:59.610989 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.193:39321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260927 18:42:59.612528 29811 catalog_manager.cc:4584] TS e231de6663c247da9546be5c98efb4d6 (127.11.251.193:39321): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bb8e06ab18614c9e91b545dea610f00e: Network error: Client connection negotiation failed: client connection to 127.11.251.193:39321: connect: Connection refused (error 111)
I20260927 18:42:59.616073 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 30147
I20260927 18:42:59.622730 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 29794
2026-09-27T18:42:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1036 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:42:59 dist-test-slave-z47j krb5kdc[30318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:42:59 dist-test-slave-z47j krb5kdc[30318](info): set up 2 sockets
Sep 27 18:42:59 dist-test-slave-z47j krb5kdc[30318](info): commencing operation
krb5kdc: starting...
W20260927 18:43:00.694006 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:00 dist-test-slave-z47j krb5kdc[30318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:43:00.833151 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:41701
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:44937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:41701
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:00.925911 30331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:00.926232 30331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:00.926273 30331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:00.927570 30331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:00.927661 30331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:00.927688 30331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:00.927723 30331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:00.927769 30331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:00.929445 30331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:44937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:41701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:41701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:00.929808 30331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:00.930097 30331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:00.935770 30336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:00.935855 30337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:00.935866 30339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:00.936053 30331 server_base.cc:1034] running on GCE node
I20260927 18:43:00.936355 30331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:00.936687 30331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:00.937863 30331 hybrid_clock.cc:648] HybridClock initialized: now 1790534580937836 us; error 49 us; skew 500 ppm
Sep 27 18:43:00 dist-test-slave-z47j krb5kdc[30318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534580, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:00.941098 30331 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:43:00.941787 30331 webserver.cc:460] Webserver started at http://127.11.251.254:35681/ using document root <none> and password file <none>
I20260927 18:43:00.942076 30331 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:00.942158 30331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:00.942339 30331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:00.943706 30331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "74edb10565ca41b7bd3125589435c799"
format_stamp: "Formatted at 2026-09-27 18:43:00 on dist-test-slave-z47j"
server_key: "6602e25f31008941c2bafc36d72f3646"
server_key_iv: "bd885113b878a738f1157c410b87800a"
server_key_version: "encryptionkey@0"
I20260927 18:43:00.944164 30331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "74edb10565ca41b7bd3125589435c799"
format_stamp: "Formatted at 2026-09-27 18:43:00 on dist-test-slave-z47j"
server_key: "6602e25f31008941c2bafc36d72f3646"
server_key_iv: "bd885113b878a738f1157c410b87800a"
server_key_version: "encryptionkey@0"
I20260927 18:43:00.945888 30331 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260927 18:43:00.946874 30346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:00.947162 30331 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:00.947299 30331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "74edb10565ca41b7bd3125589435c799"
format_stamp: "Formatted at 2026-09-27 18:43:00 on dist-test-slave-z47j"
server_key: "6602e25f31008941c2bafc36d72f3646"
server_key_iv: "bd885113b878a738f1157c410b87800a"
server_key_version: "encryptionkey@0"
I20260927 18:43:00.947392 30331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:00.971388 30331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:00.974375 30331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:00.974594 30331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:00.980512 30331 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:41701
I20260927 18:43:00.980567 30398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:41701 every 8 connection(s)
I20260927 18:43:00.981060 30331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:00.982261 30399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:00.984705 30399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799: Bootstrap starting.
I20260927 18:43:00.985272 30399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:00.985520 30399 log.cc:824] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:00.986384 30399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799: No bootstrap required, opened a new log
I20260927 18:43:00.987982 30399 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74edb10565ca41b7bd3125589435c799" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41701 } }
I20260927 18:43:00.988131 30399 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:00.988155 30399 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74edb10565ca41b7bd3125589435c799, State: Initialized, Role: FOLLOWER
I20260927 18:43:00.988268 30399 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74edb10565ca41b7bd3125589435c799" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41701 } }
I20260927 18:43:00.988371 30399 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:00.988406 30399 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:00.988452 30399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:00.989043 30399 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74edb10565ca41b7bd3125589435c799" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41701 } }
I20260927 18:43:00.989185 30399 leader_election.cc:302] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74edb10565ca41b7bd3125589435c799; no voters:
I20260927 18:43:00.989389 30399 leader_election.cc:288] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:00.989543 30402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:00.989734 30399 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:00.989820 30402 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799 [term 1 LEADER]: Becoming Leader. State: Replica: 74edb10565ca41b7bd3125589435c799, State: Running, Role: LEADER
I20260927 18:43:00.989962 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 30331
I20260927 18:43:00.990100 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:00.990062 30402 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74edb10565ca41b7bd3125589435c799" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41701 } }
I20260927 18:43:00.990363 12271 external_mini_cluster.cc:1428] Setting key 4c28c8751b2aa36be890d61cfd051c6c
I20260927 18:43:00.990979 30404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74edb10565ca41b7bd3125589435c799. Latest consensus state: current_term: 1 leader_uuid: "74edb10565ca41b7bd3125589435c799" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74edb10565ca41b7bd3125589435c799" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41701 } } }
I20260927 18:43:00.991246 30404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:00.991504 30403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74edb10565ca41b7bd3125589435c799" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74edb10565ca41b7bd3125589435c799" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41701 } } }
I20260927 18:43:00.991683 30403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:00.992103 30414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:00.992813 30414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:00.994505 30414 catalog_manager.cc:1347] Generated new cluster ID: 8be31fe3cdad4508b43b0e4e49d67a42
I20260927 18:43:00.994558 30414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:43:00 dist-test-slave-z47j krb5kdc[30318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:43:01.005400 30412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:00.990935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59938 (local address 127.11.251.254:41701)
0927 18:43:00.991164 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:00.991175 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:00.991397 (+ 222us) server_negotiation.cc:407] Connection header received
0927 18:43:00.991623 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:00.991632 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:00.991691 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:00.991810 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:00.992914 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:00.994585 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:00.996001 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:00.996769 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:00.996866 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:01.000571 (+ 3705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:01.000578 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:01.000579 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:01.000598 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:01.003107 (+ 2509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:01.003608 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:01.003612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:01.003613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:01.003760 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:01.004003 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:01.004004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:01.004005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:01.004421 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:01.004598 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:01.004917 (+ 319us) server_negotiation.cc:299] Negotiation successful
0927 18:43:01.005102 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":90,"threads_started":1}
I20260927 18:43:01.008442 30414 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:01.009030 30414 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:01.011884 30414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799: Generated new TSK 0
I20260927 18:43:01.012059 30414 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.251.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
I20260927 18:43:01.041550 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41701
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:44937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:01.129370 30425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:01.129684 30425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:01.129779 30425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:01.130988 30425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:01.131034 30425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:01.131083 30425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:43:01.132639 30425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:44937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:01.132934 30425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:01.133201 30425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:01.139101 30433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:01.139117 30431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:01.139171 30430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:01.139199 30425 server_base.cc:1034] running on GCE node
I20260927 18:43:01.139583 30425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:01.139899 30425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:01.141063 30425 hybrid_clock.cc:648] HybridClock initialized: now 1790534581141029 us; error 50 us; skew 500 ppm
Sep 27 18:43:01 dist-test-slave-z47j krb5kdc[30318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534581, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:01.144201 30425 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:43:01.144778 30425 webserver.cc:460] Webserver started at http://127.11.251.193:44513/ using document root <none> and password file <none>
I20260927 18:43:01.145066 30425 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:01.145116 30425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:01.145224 30425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:01.146523 30425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "73f9f729e26644f0be3b9d6e952e4e61"
format_stamp: "Formatted at 2026-09-27 18:43:01 on dist-test-slave-z47j"
server_key: "a44926420b603afc0ae37f4535a6d54b"
server_key_iv: "c7e9f237de12f760606158ede4292d0d"
server_key_version: "encryptionkey@0"
I20260927 18:43:01.147042 30425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "73f9f729e26644f0be3b9d6e952e4e61"
format_stamp: "Formatted at 2026-09-27 18:43:01 on dist-test-slave-z47j"
server_key: "a44926420b603afc0ae37f4535a6d54b"
server_key_iv: "c7e9f237de12f760606158ede4292d0d"
server_key_version: "encryptionkey@0"
I20260927 18:43:01.149195 30425 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260927 18:43:01.150622 30440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:01.150895 30425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:01.151067 30425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "73f9f729e26644f0be3b9d6e952e4e61"
format_stamp: "Formatted at 2026-09-27 18:43:01 on dist-test-slave-z47j"
server_key: "a44926420b603afc0ae37f4535a6d54b"
server_key_iv: "c7e9f237de12f760606158ede4292d0d"
server_key_version: "encryptionkey@0"
I20260927 18:43:01.151165 30425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:01.169180 30425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:01.171547 30425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:01.171730 30425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:01.171980 30425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:01.172329 30425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:01.172392 30425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:01.172463 30425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:01.172495 30425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:01.179859 30425 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:36315
I20260927 18:43:01.179986 30553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:36315 every 8 connection(s)
I20260927 18:43:01.180326 30425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
Sep 27 18:43:01 dist-test-slave-z47j krb5kdc[30318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534581, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:43:01.187338 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 30425
I20260927 18:43:01.187479 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:43:01.187734 12271 external_mini_cluster.cc:1428] Setting key 8e630c68214a10d620c9556f1f8cff61
I20260927 18:43:01.191921 30412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:01.181301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:59093 (local address 127.11.251.254:41701)
0927 18:43:01.181430 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:01.181434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:01.181717 (+ 283us) server_negotiation.cc:407] Connection header received
0927 18:43:01.182255 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:01.182257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:01.182269 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:01.182339 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:01.183383 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.183981 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.185029 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.185282 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.185353 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:01.188125 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:01.188128 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:01.188128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:01.188138 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:01.189900 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:01.190528 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:01.190530 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:01.190530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:01.190613 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:01.190932 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:01.190933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:01.190933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:01.191114 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:01.191212 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:01.191722 (+ 510us) server_negotiation.cc:299] Negotiation successful
0927 18:43:01.191847 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260927 18:43:01.192394 30556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:01.181246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41701 (local address 127.11.251.193:59093)
0927 18:43:01.181519 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:01.181562 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:01.182149 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:01.182401 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:01.182405 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:01.183030 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:01.183240 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:01.183245 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.184090 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.184091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:01.184923 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:01.184925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.185376 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.185377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:01.185480 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:01.186132 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:01.186140 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:01.187959 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:01.190067 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:01.190076 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:01.190079 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:01.190446 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:01.190692 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:01.190693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:01.190693 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:01.190855 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:01.191252 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:01.191261 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:01.191534 (+ 273us) client_negotiation.cc:769] Sending connection context
0927 18:43:01.191668 (+ 134us) client_negotiation.cc:240] Negotiation successful
0927 18:43:01.191886 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":163,"threads_started":1}
I20260927 18:43:01.193068 30554 heartbeater.cc:347] Connected to a master server at 127.11.251.254:41701
I20260927 18:43:01.193190 30554 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:01.193427 30554 heartbeater.cc:511] Master 127.11.251.254:41701 requested a full tablet report, sending...
I20260927 18:43:01.194106 30363 ts_manager.cc:194] Registered new tserver with Master: 73f9f729e26644f0be3b9d6e952e4e61 (127.11.251.193:36315)
I20260927 18:43:01.194788 30363 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:59093
WARNING: no policy specified for kudu/127.11.251.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
I20260927 18:43:01.217711 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41701
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:44937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:01.314509 30559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:01.314823 30559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:01.314867 30559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:01.316155 30559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:01.316231 30559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:01.316283 30559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:43:01.318008 30559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:44937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:01.318357 30559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:01.318645 30559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:01.325364 30565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:01.325330 30567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:01.325330 30564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:01.325727 30559 server_base.cc:1034] running on GCE node
I20260927 18:43:01.325971 30559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:01.326290 30559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:01.327494 30559 hybrid_clock.cc:648] HybridClock initialized: now 1790534581327473 us; error 37 us; skew 500 ppm
Sep 27 18:43:01 dist-test-slave-z47j krb5kdc[30318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534581, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:01.330612 30559 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:43:01.331221 30559 webserver.cc:460] Webserver started at http://127.11.251.194:45109/ using document root <none> and password file <none>
I20260927 18:43:01.331502 30559 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:01.331586 30559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:01.331756 30559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:01.333245 30559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "9ff00c65aec34c2d9837dd6eaef8188d"
format_stamp: "Formatted at 2026-09-27 18:43:01 on dist-test-slave-z47j"
server_key: "1201882b9cae56721d5a441effd722a2"
server_key_iv: "cd145cf6d2cff2fd19036b767836f45c"
server_key_version: "encryptionkey@0"
I20260927 18:43:01.333730 30559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "9ff00c65aec34c2d9837dd6eaef8188d"
format_stamp: "Formatted at 2026-09-27 18:43:01 on dist-test-slave-z47j"
server_key: "1201882b9cae56721d5a441effd722a2"
server_key_iv: "cd145cf6d2cff2fd19036b767836f45c"
server_key_version: "encryptionkey@0"
I20260927 18:43:01.335331 30559 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:01.336279 30574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:01.336565 30559 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:01.336687 30559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "9ff00c65aec34c2d9837dd6eaef8188d"
format_stamp: "Formatted at 2026-09-27 18:43:01 on dist-test-slave-z47j"
server_key: "1201882b9cae56721d5a441effd722a2"
server_key_iv: "cd145cf6d2cff2fd19036b767836f45c"
server_key_version: "encryptionkey@0"
I20260927 18:43:01.336771 30559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:01.359031 30559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:01.361927 30559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:01.362180 30559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:01.362540 30559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:01.363001 30559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:01.363070 30559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:01.363128 30559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:01.363183 30559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:01.371783 30559 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:44047
I20260927 18:43:01.371845 30687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:44047 every 8 connection(s)
I20260927 18:43:01.372272 30559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:43:01.373971 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 30559
I20260927 18:43:01.374115 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:43:01.374358 12271 external_mini_cluster.cc:1428] Setting key 382ba201b6847c5837706e34d5fd0888
Sep 27 18:43:01 dist-test-slave-z47j krb5kdc[30318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534581, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.251.195@KRBTEST.COM; defaulting to no policy
I20260927 18:43:01.385818 30690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:01.373162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41701 (local address 127.11.251.194:33849)
0927 18:43:01.373411 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:01.373461 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:01.374181 (+ 720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:01.374473 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:01.374477 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:01.375329 (+ 852us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:01.375652 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:01.375678 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.376664 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.376665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:01.377863 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:01.377866 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.378356 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.378358 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:01.378514 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:01.379158 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:01.379170 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:01.381434 (+ 2264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:01.383614 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:01.383627 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:01.383631 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:01.383971 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:01.384335 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:01.384338 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:01.384338 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:01.384614 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:01.384982 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:01.384990 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:01.385208 (+ 218us) client_negotiation.cc:769] Sending connection context
0927 18:43:01.385356 (+ 148us) client_negotiation.cc:240] Negotiation successful
0927 18:43:01.385544 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":114,"threads_started":1}
I20260927 18:43:01.385857 30412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:01.373262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:33849 (local address 127.11.251.254:41701)
0927 18:43:01.373412 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:01.373415 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:01.373604 (+ 189us) server_negotiation.cc:407] Connection header received
0927 18:43:01.374327 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:01.374329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:01.374343 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:01.374397 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:01.375756 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.376558 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.377988 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.378237 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.378329 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:01.381622 (+ 3293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:01.381625 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:01.381626 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:01.381636 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:01.383464 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:01.384084 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:01.384086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:01.384087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:01.384154 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:01.384728 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:01.384729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:01.384729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:01.384885 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:01.384962 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:01.385490 (+ 528us) server_negotiation.cc:299] Negotiation successful
0927 18:43:01.385771 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260927 18:43:01.386561 30688 heartbeater.cc:347] Connected to a master server at 127.11.251.254:41701
I20260927 18:43:01.386696 30688 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:01.386907 30688 heartbeater.cc:511] Master 127.11.251.254:41701 requested a full tablet report, sending...
I20260927 18:43:01.387470 30363 ts_manager.cc:194] Registered new tserver with Master: 9ff00c65aec34c2d9837dd6eaef8188d (127.11.251.194:44047)
I20260927 18:43:01.387977 30363 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:33849
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
I20260927 18:43:01.408972 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41701
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:44937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:01.504094 30693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:01.504334 30693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:01.504360 30693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:01.505730 30693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:01.505784 30693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:01.505815 30693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:43:01.507388 30693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:44937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:41701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:01.507671 30693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:01.507949 30693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:01.514461 30698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:01.514532 30701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:01.514564 30699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:01.514635 30693 server_base.cc:1034] running on GCE node
I20260927 18:43:01.514890 30693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:01.515221 30693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:01.516395 30693 hybrid_clock.cc:648] HybridClock initialized: now 1790534581516367 us; error 55 us; skew 500 ppm
Sep 27 18:43:01 dist-test-slave-z47j krb5kdc[30318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534581, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:01.523800 30693 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:43:01.524411 30693 webserver.cc:460] Webserver started at http://127.11.251.195:44957/ using document root <none> and password file <none>
I20260927 18:43:01.524674 30693 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:01.524753 30693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:01.524914 30693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:01.526173 30693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "4411eeebb4bc46e3990aa94e1c1fadb2"
format_stamp: "Formatted at 2026-09-27 18:43:01 on dist-test-slave-z47j"
server_key: "f443e4e5eb62695c28fd0af03411a0fd"
server_key_iv: "e957608527760a3102fdd1987a95957f"
server_key_version: "encryptionkey@0"
I20260927 18:43:01.526554 30693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "4411eeebb4bc46e3990aa94e1c1fadb2"
format_stamp: "Formatted at 2026-09-27 18:43:01 on dist-test-slave-z47j"
server_key: "f443e4e5eb62695c28fd0af03411a0fd"
server_key_iv: "e957608527760a3102fdd1987a95957f"
server_key_version: "encryptionkey@0"
I20260927 18:43:01.528191 30693 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:43:01.529285 30708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:01.529780 30693 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:01.529922 30693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "4411eeebb4bc46e3990aa94e1c1fadb2"
format_stamp: "Formatted at 2026-09-27 18:43:01 on dist-test-slave-z47j"
server_key: "f443e4e5eb62695c28fd0af03411a0fd"
server_key_iv: "e957608527760a3102fdd1987a95957f"
server_key_version: "encryptionkey@0"
I20260927 18:43:01.530017 30693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:01.553118 30693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:01.555775 30693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:01.555955 30693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:01.556227 30693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:01.556679 30693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:01.556744 30693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:01.556792 30693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:01.556821 30693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:01.564065 30693 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:46173
I20260927 18:43:01.564169 30821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:46173 every 8 connection(s)
I20260927 18:43:01.564559 30693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:43:01.565884 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 30693
I20260927 18:43:01.566001 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:43:01.566187 12271 external_mini_cluster.cc:1428] Setting key de69cecfc148437602d720da1e3b8ad7
Sep 27 18:43:01 dist-test-slave-z47j krb5kdc[30318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534581, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:43:01.576114 30412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:01.565552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:58583 (local address 127.11.251.254:41701)
0927 18:43:01.565676 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:01.565679 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:01.565902 (+ 223us) server_negotiation.cc:407] Connection header received
0927 18:43:01.566776 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:01.566777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:01.566791 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:01.566861 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:01.568202 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.568736 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.569894 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.570090 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.570167 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:01.572821 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:01.572823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:01.572823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:01.572832 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:01.574407 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:01.574929 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:01.574930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:01.574931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:01.574996 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:01.575352 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:01.575353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:01.575354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:01.575484 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:01.575553 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:01.575967 (+ 414us) server_negotiation.cc:299] Negotiation successful
0927 18:43:01.576040 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260927 18:43:01.576313 30824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:01.565472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:41701 (local address 127.11.251.195:58583)
0927 18:43:01.565683 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:01.565745 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:01.566598 (+ 853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:01.566987 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:01.566992 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:01.567819 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:01.568077 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:01.568082 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.568842 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.568843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:01.569771 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:01.569774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.570193 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.570194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:01.570312 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:01.570899 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:01.570910 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:01.572702 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:01.574529 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:01.574537 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:01.574539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:01.574819 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:01.575106 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:01.575107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:01.575107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:01.575234 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:01.575605 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:01.575611 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:01.575758 (+ 147us) client_negotiation.cc:769] Sending connection context
0927 18:43:01.575881 (+ 123us) client_negotiation.cc:240] Negotiation successful
0927 18:43:01.576038 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":71,"threads_started":1}
I20260927 18:43:01.576957 30822 heartbeater.cc:347] Connected to a master server at 127.11.251.254:41701
I20260927 18:43:01.577123 30822 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:01.577373 30822 heartbeater.cc:511] Master 127.11.251.254:41701 requested a full tablet report, sending...
I20260927 18:43:01.577847 30363 ts_manager.cc:194] Registered new tserver with Master: 4411eeebb4bc46e3990aa94e1c1fadb2 (127.11.251.195:46173)
I20260927 18:43:01.578509 30363 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:58583
I20260927 18:43:01.578679 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:43:01.586892 30412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:01.580287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59946 (local address 127.11.251.254:41701)
0927 18:43:01.580408 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:01.580410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:01.580529 (+ 119us) server_negotiation.cc:407] Connection header received
0927 18:43:01.580609 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:01.580610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:01.580620 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:01.580676 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:01.581262 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.581851 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.582619 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.582821 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.582912 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:01.584334 (+ 1422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:01.584336 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:01.584337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:01.584344 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:01.585540 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:01.586017 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:01.586018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:01.586019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:01.586080 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:01.586321 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:01.586322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:01.586322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:01.586464 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:01.586590 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:01.586712 (+ 122us) server_negotiation.cc:299] Negotiation successful
0927 18:43:01.586787 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260927 18:43:01.589105 30363 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59946:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:43:01.591754 30363 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:43:01.599965 30834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:01.595190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:46173 (local address 127.0.0.1:55674)
0927 18:43:01.595509 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:01.595557 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:01.595634 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:01.596053 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:01.596057 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:01.596061 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:01.596178 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:01.596181 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.597287 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.597288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:01.598457 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:01.598461 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.599557 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.599558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:01.599682 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:01.599702 (+ 20us) client_negotiation.cc:769] Sending connection context
0927 18:43:01.599798 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 18:43:01.599864 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":4,"thread_start_us":79,"threads_started":1}
I20260927 18:43:01.600039 30836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:01.595293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55674 (local address 127.11.251.195:46173)
0927 18:43:01.595634 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:01.595638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:01.595652 (+ 14us) server_negotiation.cc:407] Connection header received
0927 18:43:01.595775 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:01.595777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:01.595799 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:01.595935 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:01.596276 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.597151 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.598592 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.599441 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.599532 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:01.599599 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:01.599843 (+ 244us) server_negotiation.cc:299] Negotiation successful
0927 18:43:01.599918 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":122,"threads_started":1}
I20260927 18:43:01.600528 30835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:01.595298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56898 (local address 127.11.251.193:36315)
0927 18:43:01.595530 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:01.595537 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:01.595654 (+ 117us) server_negotiation.cc:407] Connection header received
0927 18:43:01.595841 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:01.595846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:01.595870 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:01.595970 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:01.596276 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.597274 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.599035 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.599859 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.599943 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:01.599997 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:01.600394 (+ 397us) server_negotiation.cc:299] Negotiation successful
0927 18:43:01.600453 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":94,"threads_started":1}
I20260927 18:43:01.600328 30833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:01.595195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:36315 (local address 127.0.0.1:56898)
0927 18:43:01.595559 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:01.595576 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:01.595691 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:01.596083 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:01.596084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:01.596088 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:01.596169 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:01.596171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.597403 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.597405 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:01.598917 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:01.598921 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.600043 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.600044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:01.600132 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:01.600139 (+ 7us) client_negotiation.cc:769] Sending connection context
0927 18:43:01.600197 (+ 58us) client_negotiation.cc:240] Negotiation successful
0927 18:43:01.600268 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":5,"thread_start_us":136,"threads_started":1}
I20260927 18:43:01.601456 30837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:01.595457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:44047 (local address 127.0.0.1:55166)
0927 18:43:01.595951 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:01.595963 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:01.596012 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:01.597433 (+ 1421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:01.597434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:01.597438 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:01.597491 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:01.597493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.599033 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.599034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:01.600012 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:01.600015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.601168 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.601169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:01.601242 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:01.601247 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:43:01.601318 (+ 71us) client_negotiation.cc:240] Negotiation successful
0927 18:43:01.601371 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":3,"thread_start_us":87,"threads_started":1}
I20260927 18:43:01.601536 30838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:01.595743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55166 (local address 127.11.251.194:44047)
0927 18:43:01.596004 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:01.596009 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:01.596022 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:43:01.596111 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:01.596113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:01.596132 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:01.596239 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:01.597643 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.598887 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.600118 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.600985 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.601056 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:01.601126 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:01.601351 (+ 225us) server_negotiation.cc:299] Negotiation successful
0927 18:43:01.601430 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":107,"threads_started":1}
I20260927 18:43:01.601447 30756 tablet_service.cc:1467] Processing CreateTablet for tablet 9a2a4b8f2b6e44ce93f675d7d8652319 (DEFAULT_TABLE table=test-table [id=aead4bba117442ac9f7339a84ee82766]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:01.601786 30488 tablet_service.cc:1467] Processing CreateTablet for tablet 9a2a4b8f2b6e44ce93f675d7d8652319 (DEFAULT_TABLE table=test-table [id=aead4bba117442ac9f7339a84ee82766]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:01.601958 30756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a2a4b8f2b6e44ce93f675d7d8652319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:01.602166 30488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a2a4b8f2b6e44ce93f675d7d8652319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:01.602334 30622 tablet_service.cc:1467] Processing CreateTablet for tablet 9a2a4b8f2b6e44ce93f675d7d8652319 (DEFAULT_TABLE table=test-table [id=aead4bba117442ac9f7339a84ee82766]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:01.602679 30622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a2a4b8f2b6e44ce93f675d7d8652319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:01.605214 30839 tablet_bootstrap.cc:492] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 73f9f729e26644f0be3b9d6e952e4e61: Bootstrap starting.
I20260927 18:43:01.606057 30839 tablet_bootstrap.cc:654] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 73f9f729e26644f0be3b9d6e952e4e61: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:01.606403 30840 tablet_bootstrap.cc:492] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 4411eeebb4bc46e3990aa94e1c1fadb2: Bootstrap starting.
I20260927 18:43:01.606601 30841 tablet_bootstrap.cc:492] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d: Bootstrap starting.
I20260927 18:43:01.606794 30839 log.cc:824] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 73f9f729e26644f0be3b9d6e952e4e61: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:01.607432 30841 tablet_bootstrap.cc:654] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:01.607453 30840 tablet_bootstrap.cc:654] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 4411eeebb4bc46e3990aa94e1c1fadb2: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:01.608256 30841 log.cc:824] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:01.608260 30839 tablet_bootstrap.cc:492] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 73f9f729e26644f0be3b9d6e952e4e61: No bootstrap required, opened a new log
I20260927 18:43:01.608278 30840 log.cc:824] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 4411eeebb4bc46e3990aa94e1c1fadb2: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:01.608367 30839 ts_tablet_manager.cc:1397] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 73f9f729e26644f0be3b9d6e952e4e61: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:43:01.609171 30840 tablet_bootstrap.cc:492] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 4411eeebb4bc46e3990aa94e1c1fadb2: No bootstrap required, opened a new log
I20260927 18:43:01.609198 30841 tablet_bootstrap.cc:492] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d: No bootstrap required, opened a new log
I20260927 18:43:01.609261 30841 ts_tablet_manager.cc:1397] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:43:01.609261 30840 ts_tablet_manager.cc:1397] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 4411eeebb4bc46e3990aa94e1c1fadb2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:43:01.610226 30839 raft_consensus.cc:348] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 73f9f729e26644f0be3b9d6e952e4e61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4411eeebb4bc46e3990aa94e1c1fadb2" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 46173 } } peers { permanent_uuid: "73f9f729e26644f0be3b9d6e952e4e61" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36315 } } peers { permanent_uuid: "9ff00c65aec34c2d9837dd6eaef8188d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 44047 } }
I20260927 18:43:01.610420 30839 raft_consensus.cc:374] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 73f9f729e26644f0be3b9d6e952e4e61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:01.610466 30839 raft_consensus.cc:729] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 73f9f729e26644f0be3b9d6e952e4e61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73f9f729e26644f0be3b9d6e952e4e61, State: Initialized, Role: FOLLOWER
I20260927 18:43:01.610622 30839 consensus_queue.cc:260] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 73f9f729e26644f0be3b9d6e952e4e61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4411eeebb4bc46e3990aa94e1c1fadb2" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 46173 } } peers { permanent_uuid: "73f9f729e26644f0be3b9d6e952e4e61" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36315 } } peers { permanent_uuid: "9ff00c65aec34c2d9837dd6eaef8188d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 44047 } }
I20260927 18:43:01.610939 30839 ts_tablet_manager.cc:1428] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 73f9f729e26644f0be3b9d6e952e4e61: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:43:01.611104 30554 heartbeater.cc:503] Master 127.11.251.254:41701 was elected leader, sending a full tablet report...
I20260927 18:43:01.611650 30841 raft_consensus.cc:348] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4411eeebb4bc46e3990aa94e1c1fadb2" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 46173 } } peers { permanent_uuid: "73f9f729e26644f0be3b9d6e952e4e61" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36315 } } peers { permanent_uuid: "9ff00c65aec34c2d9837dd6eaef8188d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 44047 } }
I20260927 18:43:01.611650 30840 raft_consensus.cc:348] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 4411eeebb4bc46e3990aa94e1c1fadb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4411eeebb4bc46e3990aa94e1c1fadb2" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 46173 } } peers { permanent_uuid: "73f9f729e26644f0be3b9d6e952e4e61" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36315 } } peers { permanent_uuid: "9ff00c65aec34c2d9837dd6eaef8188d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 44047 } }
I20260927 18:43:01.611866 30840 raft_consensus.cc:374] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 4411eeebb4bc46e3990aa94e1c1fadb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:01.611866 30841 raft_consensus.cc:374] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:01.611922 30840 raft_consensus.cc:729] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 4411eeebb4bc46e3990aa94e1c1fadb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4411eeebb4bc46e3990aa94e1c1fadb2, State: Initialized, Role: FOLLOWER
I20260927 18:43:01.611922 30841 raft_consensus.cc:729] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ff00c65aec34c2d9837dd6eaef8188d, State: Initialized, Role: FOLLOWER
I20260927 18:43:01.612088 30841 consensus_queue.cc:260] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4411eeebb4bc46e3990aa94e1c1fadb2" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 46173 } } peers { permanent_uuid: "73f9f729e26644f0be3b9d6e952e4e61" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36315 } } peers { permanent_uuid: "9ff00c65aec34c2d9837dd6eaef8188d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 44047 } }
I20260927 18:43:01.612088 30840 consensus_queue.cc:260] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 4411eeebb4bc46e3990aa94e1c1fadb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4411eeebb4bc46e3990aa94e1c1fadb2" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 46173 } } peers { permanent_uuid: "73f9f729e26644f0be3b9d6e952e4e61" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36315 } } peers { permanent_uuid: "9ff00c65aec34c2d9837dd6eaef8188d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 44047 } }
I20260927 18:43:01.612427 30841 ts_tablet_manager.cc:1428] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:43:01.612433 30840 ts_tablet_manager.cc:1428] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 4411eeebb4bc46e3990aa94e1c1fadb2: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260927 18:43:01.612572 30822 heartbeater.cc:503] Master 127.11.251.254:41701 was elected leader, sending a full tablet report...
I20260927 18:43:01.612572 30688 heartbeater.cc:503] Master 127.11.251.254:41701 was elected leader, sending a full tablet report...
W20260927 18:43:01.623092 30689 tablet.cc:2369] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:43:01.660879 30847 raft_consensus.cc:482] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:43:01.661095 30847 raft_consensus.cc:504] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4411eeebb4bc46e3990aa94e1c1fadb2" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 46173 } } peers { permanent_uuid: "73f9f729e26644f0be3b9d6e952e4e61" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36315 } } peers { permanent_uuid: "9ff00c65aec34c2d9837dd6eaef8188d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 44047 } }
I20260927 18:43:01.661615 30847 leader_election.cc:288] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4411eeebb4bc46e3990aa94e1c1fadb2 (127.11.251.195:46173), 73f9f729e26644f0be3b9d6e952e4e61 (127.11.251.193:36315)
I20260927 18:43:01.664880 30836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:01.661991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:59079 (local address 127.11.251.195:46173)
0927 18:43:01.662075 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:01.662077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:01.662090 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:43:01.662171 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:01.662173 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:01.662183 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:01.662263 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:01.662465 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.663001 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.664029 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.664453 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.664573 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:01.664585 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:01.664755 (+ 170us) server_negotiation.cc:299] Negotiation successful
0927 18:43:01.664799 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 18:43:01.664887 30690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:01.661869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:46173 (local address 127.11.251.194:59079)
0927 18:43:01.661958 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:01.661974 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:01.662089 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:01.662267 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:01.662268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:01.662270 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:01.662365 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:01.662368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.663095 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.663096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:01.663928 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:01.663930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.664569 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.664570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:01.664669 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:01.664673 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:43:01.664747 (+ 74us) client_negotiation.cc:240] Negotiation successful
0927 18:43:01.664799 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260927 18:43:01.665246 30848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:01.661869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:36315 (local address 127.11.251.194:33349)
0927 18:43:01.662263 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:01.662280 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:01.662374 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:01.662549 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:01.662550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:01.662551 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:01.662646 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:01.662648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.663337 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.663338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:01.664125 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:01.664127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.664815 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.664816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:01.664901 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:01.664907 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:43:01.665070 (+ 163us) client_negotiation.cc:240] Negotiation successful
0927 18:43:01.665131 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":166,"thread_start_us":154,"threads_started":1}
I20260927 18:43:01.665266 30835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:01.662008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:33349 (local address 127.11.251.193:36315)
0927 18:43:01.662073 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:01.662076 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:01.662354 (+ 278us) server_negotiation.cc:407] Connection header received
0927 18:43:01.662461 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:01.662462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:01.662473 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:01.662528 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:01.662732 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.663244 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.664282 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.664723 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.664815 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:01.664825 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:01.665158 (+ 333us) server_negotiation.cc:299] Negotiation successful
0927 18:43:01.665201 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 18:43:01.665311 30776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a2a4b8f2b6e44ce93f675d7d8652319" candidate_uuid: "9ff00c65aec34c2d9837dd6eaef8188d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4411eeebb4bc46e3990aa94e1c1fadb2" is_pre_election: true
I20260927 18:43:01.665510 30776 raft_consensus.cc:2454] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 4411eeebb4bc46e3990aa94e1c1fadb2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ff00c65aec34c2d9837dd6eaef8188d in term 0.
I20260927 18:43:01.665647 30508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a2a4b8f2b6e44ce93f675d7d8652319" candidate_uuid: "9ff00c65aec34c2d9837dd6eaef8188d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73f9f729e26644f0be3b9d6e952e4e61" is_pre_election: true
I20260927 18:43:01.665843 30577 leader_election.cc:302] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4411eeebb4bc46e3990aa94e1c1fadb2, 9ff00c65aec34c2d9837dd6eaef8188d; no voters:
I20260927 18:43:01.665884 30508 raft_consensus.cc:2454] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 73f9f729e26644f0be3b9d6e952e4e61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ff00c65aec34c2d9837dd6eaef8188d in term 0.
I20260927 18:43:01.666023 30847 raft_consensus.cc:2781] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:43:01.666114 30847 raft_consensus.cc:482] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:43:01.666152 30847 raft_consensus.cc:3037] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:01.667057 30847 raft_consensus.cc:504] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4411eeebb4bc46e3990aa94e1c1fadb2" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 46173 } } peers { permanent_uuid: "73f9f729e26644f0be3b9d6e952e4e61" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36315 } } peers { permanent_uuid: "9ff00c65aec34c2d9837dd6eaef8188d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 44047 } }
I20260927 18:43:01.667237 30847 leader_election.cc:288] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d [CANDIDATE]: Term 1 election: Requested vote from peers 4411eeebb4bc46e3990aa94e1c1fadb2 (127.11.251.195:46173), 73f9f729e26644f0be3b9d6e952e4e61 (127.11.251.193:36315)
I20260927 18:43:01.667471 30508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a2a4b8f2b6e44ce93f675d7d8652319" candidate_uuid: "9ff00c65aec34c2d9837dd6eaef8188d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73f9f729e26644f0be3b9d6e952e4e61"
I20260927 18:43:01.667471 30776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a2a4b8f2b6e44ce93f675d7d8652319" candidate_uuid: "9ff00c65aec34c2d9837dd6eaef8188d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4411eeebb4bc46e3990aa94e1c1fadb2"
I20260927 18:43:01.667567 30508 raft_consensus.cc:3037] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 73f9f729e26644f0be3b9d6e952e4e61 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:01.667577 30776 raft_consensus.cc:3037] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 4411eeebb4bc46e3990aa94e1c1fadb2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:01.668622 30776 raft_consensus.cc:2454] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 4411eeebb4bc46e3990aa94e1c1fadb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ff00c65aec34c2d9837dd6eaef8188d in term 1.
I20260927 18:43:01.668617 30508 raft_consensus.cc:2454] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 73f9f729e26644f0be3b9d6e952e4e61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ff00c65aec34c2d9837dd6eaef8188d in term 1.
I20260927 18:43:01.668867 30577 leader_election.cc:302] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4411eeebb4bc46e3990aa94e1c1fadb2, 9ff00c65aec34c2d9837dd6eaef8188d; no voters:
I20260927 18:43:01.669015 30847 raft_consensus.cc:2781] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:01.669196 30847 raft_consensus.cc:686] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d [term 1 LEADER]: Becoming Leader. State: Replica: 9ff00c65aec34c2d9837dd6eaef8188d, State: Running, Role: LEADER
I20260927 18:43:01.669337 30847 consensus_queue.cc:237] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4411eeebb4bc46e3990aa94e1c1fadb2" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 46173 } } peers { permanent_uuid: "73f9f729e26644f0be3b9d6e952e4e61" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36315 } } peers { permanent_uuid: "9ff00c65aec34c2d9837dd6eaef8188d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 44047 } }
I20260927 18:43:01.670428 30363 catalog_manager.cc:5495] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ff00c65aec34c2d9837dd6eaef8188d (127.11.251.194). New cstate: current_term: 1 leader_uuid: "9ff00c65aec34c2d9837dd6eaef8188d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4411eeebb4bc46e3990aa94e1c1fadb2" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 46173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73f9f729e26644f0be3b9d6e952e4e61" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ff00c65aec34c2d9837dd6eaef8188d" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 44047 } health_report { overall_health: HEALTHY } } }
W20260927 18:43:01.681298 30555 tablet.cc:2369] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 73f9f729e26644f0be3b9d6e952e4e61: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:43:01.696300 30838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:01.693545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55170 (local address 127.11.251.194:44047)
0927 18:43:01.693680 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:01.693682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:01.693692 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:43:01.693738 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:01.693740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:01.693763 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:01.693843 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:01.694007 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.694555 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.695451 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.695649 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.695719 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:01.695888 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:01.695995 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:01.696081 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:01.696203 (+ 122us) server_negotiation.cc:299] Negotiation successful
0927 18:43:01.696217 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260927 18:43:01.698833 30508 raft_consensus.cc:1261] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 73f9f729e26644f0be3b9d6e952e4e61 [term 1 FOLLOWER]: Refusing update from remote peer 9ff00c65aec34c2d9837dd6eaef8188d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:01.698848 30776 raft_consensus.cc:1261] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 4411eeebb4bc46e3990aa94e1c1fadb2 [term 1 FOLLOWER]: Refusing update from remote peer 9ff00c65aec34c2d9837dd6eaef8188d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:01.699283 30849 consensus_queue.cc:1035] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d [LEADER]: Connected to new peer: Peer: permanent_uuid: "73f9f729e26644f0be3b9d6e952e4e61" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 36315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:01.699510 30847 consensus_queue.cc:1035] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4411eeebb4bc46e3990aa94e1c1fadb2" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 46173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:01.705694 30362 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:43:01.705828 30362 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:43:01.706859 30362 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74edb10565ca41b7bd3125589435c799: Sending DeleteTablet for 3 replicas of tablet 9a2a4b8f2b6e44ce93f675d7d8652319
I20260927 18:43:01.707285 30756 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a2a4b8f2b6e44ce93f675d7d8652319 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:01 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:55674
I20260927 18:43:01.707646 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 30425
I20260927 18:43:01.707707 30862 tablet_replica.cc:330] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 4411eeebb4bc46e3990aa94e1c1fadb2: stopping tablet replica
I20260927 18:43:01.708320 30862 raft_consensus.cc:2229] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 4411eeebb4bc46e3990aa94e1c1fadb2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:43:01.708487 30862 raft_consensus.cc:2258] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 4411eeebb4bc46e3990aa94e1c1fadb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:01.709910 30622 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a2a4b8f2b6e44ce93f675d7d8652319 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:01 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:55166
I20260927 18:43:01.710182 30863 tablet_replica.cc:330] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d: stopping tablet replica
I20260927 18:43:01.710309 30863 raft_consensus.cc:2229] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:43:01.710448 30863 raft_consensus.cc:2258] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:01.710820 30862 ts_tablet_manager.cc:1905] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 4411eeebb4bc46e3990aa94e1c1fadb2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:01.711025 30863 ts_tablet_manager.cc:1905] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:01.712834 30862 ts_tablet_manager.cc:1918] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 4411eeebb4bc46e3990aa94e1c1fadb2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:43:01.712996 30862 log.cc:1196] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 4411eeebb4bc46e3990aa94e1c1fadb2: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/9a2a4b8f2b6e44ce93f675d7d8652319
I20260927 18:43:01.713370 30862 ts_tablet_manager.cc:1939] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 4411eeebb4bc46e3990aa94e1c1fadb2: Deleting consensus metadata
I20260927 18:43:01.713723 30863 ts_tablet_manager.cc:1918] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:43:01.713819 30863 log.cc:1196] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/9a2a4b8f2b6e44ce93f675d7d8652319
I20260927 18:43:01.714149 30863 ts_tablet_manager.cc:1939] T 9a2a4b8f2b6e44ce93f675d7d8652319 P 9ff00c65aec34c2d9837dd6eaef8188d: Deleting consensus metadata
I20260927 18:43:01.714167 30349 catalog_manager.cc:4841] TS 4411eeebb4bc46e3990aa94e1c1fadb2 (127.11.251.195:46173): tablet 9a2a4b8f2b6e44ce93f675d7d8652319 (table test-table [id=aead4bba117442ac9f7339a84ee82766]) successfully deleted
I20260927 18:43:01.714697 30348 catalog_manager.cc:4841] TS 9ff00c65aec34c2d9837dd6eaef8188d (127.11.251.194:44047): tablet 9a2a4b8f2b6e44ce93f675d7d8652319 (table test-table [id=aead4bba117442ac9f7339a84ee82766]) successfully deleted
W20260927 18:43:01.716888 30348 connection.cc:535] client connection to 127.11.251.193:36315 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 18:43:01.717012 30348 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 18:43:01.718216 30833 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:01.718043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:36315 (local address 127.0.0.1:56902)
0927 18:43:01.718132 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:01.718178 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.193:36315: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":60}
W20260927 18:43:01.718374 30348 catalog_manager.cc:4584] TS 73f9f729e26644f0be3b9d6e952e4e61 (127.11.251.193:36315): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9a2a4b8f2b6e44ce93f675d7d8652319: Network error: Client connection negotiation failed: client connection to 127.11.251.193:36315: connect: Connection reset by peer (error 104)
I20260927 18:43:01.718703 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 30559
I20260927 18:43:01.724298 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 30693
I20260927 18:43:01.730075 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 30331
2026-09-27T18:43:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2108 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-27T18:43:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:01Z Disabled control of system clock
I20260927 18:43:01.761224 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:38349
--webserver_interface=127.11.251.254
--webserver_port=0
--builtin_ntp_servers=127.11.251.212:43545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:38349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:43:01.852540 30868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:01.852823 30868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:01.852847 30868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:01.854063 30868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:01.854110 30868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:01.854125 30868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:01.854141 30868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:01.854171 30868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:01.855581 30868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:43545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:38349
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:38349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:01.855821 30868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:01.856172 30868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:01.861681 30876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:01.861685 30874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:01.861685 30873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:01.862181 30868 server_base.cc:1034] running on GCE node
I20260927 18:43:01.862524 30868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:01.862855 30868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:01.864037 30868 hybrid_clock.cc:648] HybridClock initialized: now 1790534581864024 us; error 36 us; skew 500 ppm
I20260927 18:43:01.865845 30868 webserver.cc:460] Webserver started at http://127.11.251.254:38393/ using document root <none> and password file <none>
I20260927 18:43:01.866196 30868 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:01.866257 30868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:01.866437 30868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:01.867799 30868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "57131633effa49b68ef0667b2ae09a9f"
format_stamp: "Formatted at 2026-09-27 18:43:01 on dist-test-slave-z47j"
server_key: "28aea8dbef524002b805945b16036ac0"
server_key_iv: "9edbd5291a01a8436fa6f5872237abe1"
server_key_version: "encryptionkey@0"
I20260927 18:43:01.868250 30868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "57131633effa49b68ef0667b2ae09a9f"
format_stamp: "Formatted at 2026-09-27 18:43:01 on dist-test-slave-z47j"
server_key: "28aea8dbef524002b805945b16036ac0"
server_key_iv: "9edbd5291a01a8436fa6f5872237abe1"
server_key_version: "encryptionkey@0"
I20260927 18:43:01.870020 30868 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260927 18:43:01.871106 30882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:01.871469 30868 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:01.871603 30868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "57131633effa49b68ef0667b2ae09a9f"
format_stamp: "Formatted at 2026-09-27 18:43:01 on dist-test-slave-z47j"
server_key: "28aea8dbef524002b805945b16036ac0"
server_key_iv: "9edbd5291a01a8436fa6f5872237abe1"
server_key_version: "encryptionkey@0"
I20260927 18:43:01.871698 30868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:01.903083 30868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:01.903582 30868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:01.903784 30868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:01.909847 30868 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:38349
I20260927 18:43:01.909958 30934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:38349 every 8 connection(s)
I20260927 18:43:01.910403 30868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:01.911644 30935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:01.914575 30935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f: Bootstrap starting.
I20260927 18:43:01.915262 30935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:01.915616 30935 log.cc:824] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:01.916576 30935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f: No bootstrap required, opened a new log
I20260927 18:43:01.917637 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 30868
I20260927 18:43:01.917768 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:01.918007 12271 external_mini_cluster.cc:1428] Setting key 028482f1c5786a28922fbe713c2940ea
I20260927 18:43:01.919224 30935 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57131633effa49b68ef0667b2ae09a9f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 38349 } }
I20260927 18:43:01.919423 30935 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:01.919466 30935 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57131633effa49b68ef0667b2ae09a9f, State: Initialized, Role: FOLLOWER
I20260927 18:43:01.919621 30935 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57131633effa49b68ef0667b2ae09a9f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 38349 } }
I20260927 18:43:01.919744 30935 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:01.919799 30935 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:01.919873 30935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:01.920552 30935 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57131633effa49b68ef0667b2ae09a9f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 38349 } }
I20260927 18:43:01.920648 30935 leader_election.cc:302] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57131633effa49b68ef0667b2ae09a9f; no voters:
I20260927 18:43:01.920882 30935 leader_election.cc:288] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:01.920922 30940 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:01.921057 30940 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f [term 1 LEADER]: Becoming Leader. State: Replica: 57131633effa49b68ef0667b2ae09a9f, State: Running, Role: LEADER
I20260927 18:43:01.921190 30935 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:01.921154 30940 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57131633effa49b68ef0667b2ae09a9f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 38349 } }
I20260927 18:43:01.921554 30940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57131633effa49b68ef0667b2ae09a9f. Latest consensus state: current_term: 1 leader_uuid: "57131633effa49b68ef0667b2ae09a9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57131633effa49b68ef0667b2ae09a9f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 38349 } } }
I20260927 18:43:01.921675 30940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:01.921602 30942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57131633effa49b68ef0667b2ae09a9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57131633effa49b68ef0667b2ae09a9f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 38349 } } }
I20260927 18:43:01.921942 30942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:01.922067 30945 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:01.922683 30945 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:01.924940 30945 catalog_manager.cc:1347] Generated new cluster ID: 728a49536abb4b8a852579c80f3dbf16
I20260927 18:43:01.924999 30945 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:01.925698 30939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:01.918767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39362 (local address 127.11.251.254:38349)
0927 18:43:01.919067 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:01.919077 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:01.919111 (+ 34us) server_negotiation.cc:407] Connection header received
0927 18:43:01.919251 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:01.919258 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:01.919328 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:01.919517 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:01.919821 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.921678 (+ 1857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.922876 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:01.923522 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:01.924066 (+ 544us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:01.924652 (+ 586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:01.924656 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:01.924656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:01.924920 (+ 264us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:01.924978 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:01.924996 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:01.925279 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:01.925315 (+ 36us) server_negotiation.cc:299] Negotiation successful
0927 18:43:01.925408 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":171,"threads_started":1}
I20260927 18:43:01.930991 30945 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:01.931792 30945 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:01.940132 30945 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f: Generated new TSK 0
I20260927 18:43:01.940419 30945 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:01.943888 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:38349
--builtin_ntp_servers=127.11.251.212:43545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:43:02.041232 30959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:02.041515 30959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:02.041539 30959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:02.043046 30959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:02.043093 30959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:02.043123 30959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:43:02.044684 30959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:43545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:38349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:02.044981 30959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:02.045375 30959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:02.051265 30964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:02.051501 30967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:02.051312 30965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:02.051537 30959 server_base.cc:1034] running on GCE node
I20260927 18:43:02.051826 30959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:02.052122 30959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:02.053360 30959 hybrid_clock.cc:648] HybridClock initialized: now 1790534582053328 us; error 56 us; skew 500 ppm
I20260927 18:43:02.054993 30959 webserver.cc:460] Webserver started at http://127.11.251.193:35663/ using document root <none> and password file <none>
I20260927 18:43:02.055322 30959 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:02.055408 30959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:02.055585 30959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:02.056854 30959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "ad2a4b5b86fc46e785fd428d348cab88"
format_stamp: "Formatted at 2026-09-27 18:43:02 on dist-test-slave-z47j"
server_key: "05e6bf244bf71a42336cc9d1c46d9c8e"
server_key_iv: "1ef59fd1b1a81fae3e70564a371c45fd"
server_key_version: "encryptionkey@0"
I20260927 18:43:02.057283 30959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "ad2a4b5b86fc46e785fd428d348cab88"
format_stamp: "Formatted at 2026-09-27 18:43:02 on dist-test-slave-z47j"
server_key: "05e6bf244bf71a42336cc9d1c46d9c8e"
server_key_iv: "1ef59fd1b1a81fae3e70564a371c45fd"
server_key_version: "encryptionkey@0"
I20260927 18:43:02.058966 30959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260927 18:43:02.059957 30973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:02.060257 30959 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:02.060375 30959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "ad2a4b5b86fc46e785fd428d348cab88"
format_stamp: "Formatted at 2026-09-27 18:43:02 on dist-test-slave-z47j"
server_key: "05e6bf244bf71a42336cc9d1c46d9c8e"
server_key_iv: "1ef59fd1b1a81fae3e70564a371c45fd"
server_key_version: "encryptionkey@0"
I20260927 18:43:02.060482 30959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:02.068841 30959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:02.069252 30959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:02.069427 30959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:02.069761 30959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:02.070159 30959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:02.070221 30959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:02.070268 30959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:02.070295 30959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:02.078164 30959 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:38357
I20260927 18:43:02.078218 31086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:38357 every 8 connection(s)
I20260927 18:43:02.078608 30959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:43:02.083830 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 30959
I20260927 18:43:02.083984 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:43:02.084229 12271 external_mini_cluster.cc:1428] Setting key 2fcc950e61dd30681946e3fbee47b6a4
I20260927 18:43:02.085246 30939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:02.079458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:37059 (local address 127.11.251.254:38349)
0927 18:43:02.079537 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:02.079544 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:02.079817 (+ 273us) server_negotiation.cc:407] Connection header received
0927 18:43:02.080361 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:02.080369 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:02.080384 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:02.080483 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:02.081348 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.082116 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.083229 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.083471 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.083548 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:02.084847 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:02.084849 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:02.084849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:02.084855 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:02.084877 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:02.084881 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:02.084943 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:02.085090 (+ 147us) server_negotiation.cc:299] Negotiation successful
0927 18:43:02.085144 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:43:02.085557 31089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:02.079449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:38349 (local address 127.11.251.193:37059)
0927 18:43:02.079666 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:02.079711 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:02.080251 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:02.080597 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:02.080602 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:02.080619 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:02.081213 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:02.081219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.082258 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.082259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:02.083119 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:02.083121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.083606 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.083608 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:02.083805 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:02.084655 (+ 850us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:43:02.084656 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:02.084698 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:43:02.084730 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:02.084960 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:02.084966 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:02.084968 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:43:02.085080 (+ 112us) client_negotiation.cc:240] Negotiation successful
0927 18:43:02.085265 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":115,"threads_started":1}
I20260927 18:43:02.085688 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:38349
--builtin_ntp_servers=127.11.251.212:43545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 18:43:02.086251 31087 heartbeater.cc:347] Connected to a master server at 127.11.251.254:38349
I20260927 18:43:02.086374 31087 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:02.086620 31087 heartbeater.cc:511] Master 127.11.251.254:38349 requested a full tablet report, sending...
I20260927 18:43:02.087282 30899 ts_manager.cc:194] Registered new tserver with Master: ad2a4b5b86fc46e785fd428d348cab88 (127.11.251.193:38357)
I20260927 18:43:02.088049 30899 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.251.193:37059
W20260927 18:43:02.178434 31090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:02.178685 31090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:02.178711 31090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:02.180199 31090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:02.180258 31090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:02.180287 31090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:43:02.182093 31090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:43545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:38349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:02.182401 31090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:02.182772 31090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:02.188684 31098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:02.188798 31090 server_base.cc:1034] running on GCE node
W20260927 18:43:02.188699 31096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:02.188694 31095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:02.189265 31090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:02.189560 31090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:02.190810 31090 hybrid_clock.cc:648] HybridClock initialized: now 1790534582190781 us; error 48 us; skew 500 ppm
I20260927 18:43:02.192329 31090 webserver.cc:460] Webserver started at http://127.11.251.194:45431/ using document root <none> and password file <none>
I20260927 18:43:02.192703 31090 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:02.192787 31090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:02.192955 31090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:02.194128 31090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "e06b9968322d4cc1ae348d699313dbee"
format_stamp: "Formatted at 2026-09-27 18:43:02 on dist-test-slave-z47j"
server_key: "bc2b1fe9b09db97270e685b8028c9425"
server_key_iv: "94ccc01c30221c2541eff8f52e127ddf"
server_key_version: "encryptionkey@0"
I20260927 18:43:02.194559 31090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "e06b9968322d4cc1ae348d699313dbee"
format_stamp: "Formatted at 2026-09-27 18:43:02 on dist-test-slave-z47j"
server_key: "bc2b1fe9b09db97270e685b8028c9425"
server_key_iv: "94ccc01c30221c2541eff8f52e127ddf"
server_key_version: "encryptionkey@0"
I20260927 18:43:02.196055 31090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260927 18:43:02.196955 31104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:02.197273 31090 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:02.197413 31090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "e06b9968322d4cc1ae348d699313dbee"
format_stamp: "Formatted at 2026-09-27 18:43:02 on dist-test-slave-z47j"
server_key: "bc2b1fe9b09db97270e685b8028c9425"
server_key_iv: "94ccc01c30221c2541eff8f52e127ddf"
server_key_version: "encryptionkey@0"
I20260927 18:43:02.197511 31090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:02.211498 31090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:02.211823 31090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:02.211982 31090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:02.212234 31090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:02.212594 31090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:02.212663 31090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:02.212706 31090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:02.212735 31090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:02.219739 31090 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:41959
I20260927 18:43:02.219826 31217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:41959 every 8 connection(s)
I20260927 18:43:02.220209 31090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:43:02.221619 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 31090
I20260927 18:43:02.221743 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:43:02.222003 12271 external_mini_cluster.cc:1428] Setting key 960135c39ab793585accaf9228a6be0f
I20260927 18:43:02.223237 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:38349
--builtin_ntp_servers=127.11.251.212:43545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 18:43:02.227804 30939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:02.221336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:56101 (local address 127.11.251.254:38349)
0927 18:43:02.221468 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:02.221472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:02.222030 (+ 558us) server_negotiation.cc:407] Connection header received
0927 18:43:02.222766 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:02.222767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:02.222781 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:02.222844 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:02.223717 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.224554 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.226143 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.226343 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.226417 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:02.227454 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:02.227456 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:02.227457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:02.227462 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:02.227483 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:02.227489 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:02.227555 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:02.227671 (+ 116us) server_negotiation.cc:299] Negotiation successful
0927 18:43:02.227718 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260927 18:43:02.228010 31220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:02.221290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:38349 (local address 127.11.251.194:56101)
0927 18:43:02.221809 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:02.221864 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:02.222545 (+ 681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:02.222948 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:02.222952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:02.222963 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:02.223604 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:02.223609 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.224687 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.224688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:02.226031 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:02.226036 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.226435 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.226436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:02.226552 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:02.227267 (+ 715us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:43:02.227268 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:02.227297 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:43:02.227321 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:02.227555 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:02.227562 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:02.227565 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:43:02.227671 (+ 106us) client_negotiation.cc:240] Negotiation successful
0927 18:43:02.227776 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":209,"threads_started":1}
I20260927 18:43:02.228514 31218 heartbeater.cc:347] Connected to a master server at 127.11.251.254:38349
I20260927 18:43:02.228614 31218 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:02.228825 31218 heartbeater.cc:511] Master 127.11.251.254:38349 requested a full tablet report, sending...
I20260927 18:43:02.229264 30899 ts_manager.cc:194] Registered new tserver with Master: e06b9968322d4cc1ae348d699313dbee (127.11.251.194:41959)
I20260927 18:43:02.229751 30899 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.251.194:56101
W20260927 18:43:02.316479 31221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:02.316766 31221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:02.316802 31221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:02.318019 31221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:02.318090 31221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:02.318137 31221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:43:02.319634 31221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:43545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:38349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:02.319945 31221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:02.320286 31221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:02.326126 31229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:02.326045 31227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:02.326045 31226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:02.326315 31221 server_base.cc:1034] running on GCE node
I20260927 18:43:02.326579 31221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:02.326938 31221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:02.328140 31221 hybrid_clock.cc:648] HybridClock initialized: now 1790534582328115 us; error 48 us; skew 500 ppm
I20260927 18:43:02.329846 31221 webserver.cc:460] Webserver started at http://127.11.251.195:36731/ using document root <none> and password file <none>
I20260927 18:43:02.330178 31221 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:02.330240 31221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:02.330402 31221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:02.331773 31221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "713f06dbc2d44e2ca6fb88eeb03c95a3"
format_stamp: "Formatted at 2026-09-27 18:43:02 on dist-test-slave-z47j"
server_key: "a8db069a6b8eb0a9769e773c9267346c"
server_key_iv: "4371ef5d55f9e69efa5cd106436f2217"
server_key_version: "encryptionkey@0"
I20260927 18:43:02.332190 31221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "713f06dbc2d44e2ca6fb88eeb03c95a3"
format_stamp: "Formatted at 2026-09-27 18:43:02 on dist-test-slave-z47j"
server_key: "a8db069a6b8eb0a9769e773c9267346c"
server_key_iv: "4371ef5d55f9e69efa5cd106436f2217"
server_key_version: "encryptionkey@0"
I20260927 18:43:02.333967 31221 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260927 18:43:02.334950 31235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:02.335219 31221 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:02.335337 31221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "713f06dbc2d44e2ca6fb88eeb03c95a3"
format_stamp: "Formatted at 2026-09-27 18:43:02 on dist-test-slave-z47j"
server_key: "a8db069a6b8eb0a9769e773c9267346c"
server_key_iv: "4371ef5d55f9e69efa5cd106436f2217"
server_key_version: "encryptionkey@0"
I20260927 18:43:02.335428 31221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:02.348747 31221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:02.349097 31221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:02.349256 31221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:02.349496 31221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:02.349887 31221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:02.349951 31221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:02.349990 31221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:02.350018 31221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:02.356983 31221 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:44811
I20260927 18:43:02.357118 31348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:44811 every 8 connection(s)
I20260927 18:43:02.357446 31221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:43:02.360713 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 31221
I20260927 18:43:02.360836 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:43:02.361063 12271 external_mini_cluster.cc:1428] Setting key 82f12cb041a49a835cb45d16b84d1e46
I20260927 18:43:02.364579 30939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:02.358434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:42351 (local address 127.11.251.254:38349)
0927 18:43:02.358522 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:02.358526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:02.358915 (+ 389us) server_negotiation.cc:407] Connection header received
0927 18:43:02.359596 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:02.359598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:02.359610 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:02.359692 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:02.360638 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.361422 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.362943 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.363223 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.363292 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:02.364347 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:02.364348 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:02.364348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:02.364353 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:02.364368 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:02.364372 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:02.364419 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:02.364492 (+ 73us) server_negotiation.cc:299] Negotiation successful
0927 18:43:02.364517 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 18:43:02.364907 31351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:02.358413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:38349 (local address 127.11.251.195:42351)
0927 18:43:02.358712 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:02.358760 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:02.359469 (+ 709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:02.359813 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:02.359818 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:02.359830 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:02.360504 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:02.360510 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.361568 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.361569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:02.362810 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:02.362813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.363321 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.363322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:02.363465 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:02.364193 (+ 728us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:43:02.364194 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:02.364230 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:43:02.364258 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:02.364431 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:02.364434 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:02.364437 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:43:02.364495 (+ 58us) client_negotiation.cc:240] Negotiation successful
0927 18:43:02.364592 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":125,"threads_started":1}
I20260927 18:43:02.365720 31349 heartbeater.cc:347] Connected to a master server at 127.11.251.254:38349
I20260927 18:43:02.365840 31349 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:02.366077 31349 heartbeater.cc:511] Master 127.11.251.254:38349 requested a full tablet report, sending...
I20260927 18:43:02.366580 30899 ts_manager.cc:194] Registered new tserver with Master: 713f06dbc2d44e2ca6fb88eeb03c95a3 (127.11.251.195:44811)
I20260927 18:43:02.367056 30899 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.251.195:42351
I20260927 18:43:02.373445 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:43:02.378505 30939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:02.375098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39368 (local address 127.11.251.254:38349)
0927 18:43:02.375192 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:02.375195 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:02.375326 (+ 131us) server_negotiation.cc:407] Connection header received
0927 18:43:02.375375 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:02.375376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:02.375391 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:02.375446 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:02.375718 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.376336 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.377097 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.377287 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.377346 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:02.378066 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:02.378068 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:02.378068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:02.378072 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:02.378089 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:02.378093 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:02.378174 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:02.378311 (+ 137us) server_negotiation.cc:299] Negotiation successful
0927 18:43:02.378369 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 18:43:02.380304 30899 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39368:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:43:02.382357 30899 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:43:02.390067 31360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:02.385777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:41959 (local address 127.0.0.1:54864)
0927 18:43:02.386166 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:02.386177 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:02.386232 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:02.386461 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:02.386462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:02.386465 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:02.386528 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:02.386531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.387447 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.387448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:02.388693 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:02.388696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.389733 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.389734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:02.389817 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:02.389833 (+ 16us) client_negotiation.cc:769] Sending connection context
0927 18:43:02.389913 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 18:43:02.389973 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":127,"threads_started":1}
I20260927 18:43:02.390086 31361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:02.385968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54864 (local address 127.11.251.194:41959)
0927 18:43:02.386158 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:02.386163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:02.386221 (+ 58us) server_negotiation.cc:407] Connection header received
0927 18:43:02.386298 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:02.386301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:02.386317 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:02.386393 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:02.386612 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.387353 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.388783 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.389560 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.389713 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:02.389823 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:02.389929 (+ 106us) server_negotiation.cc:299] Negotiation successful
0927 18:43:02.389979 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
I20260927 18:43:02.390448 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:02.386058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35004 (local address 127.11.251.193:38357)
0927 18:43:02.386369 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:02.386374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:02.386384 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:43:02.386428 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:02.386431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:02.386447 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:02.386683 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:02.386737 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.387826 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.389069 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.389807 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.390263 (+ 456us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:02.390325 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:02.390345 (+ 20us) server_negotiation.cc:299] Negotiation successful
0927 18:43:02.390383 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":102,"threads_started":1}
I20260927 18:43:02.390185 31357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:02.385571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:38357 (local address 127.0.0.1:35004)
0927 18:43:02.385921 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:02.385934 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:02.386001 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:02.386561 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:02.386562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:02.386565 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:02.386610 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:02.386611 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.387923 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.387924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:02.388967 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:02.388968 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.389911 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.389912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:02.389996 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:02.390002 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:43:02.390035 (+ 33us) client_negotiation.cc:240] Negotiation successful
0927 18:43:02.390098 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":30,"thread_start_us":96,"threads_started":1}
I20260927 18:43:02.391129 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:02.385456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:44811 (local address 127.0.0.1:36002)
0927 18:43:02.385730 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:02.385779 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:02.385892 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:02.386335 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:02.386338 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:02.386342 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:02.386459 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:02.386461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.387923 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.387924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:02.389721 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:02.389724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.390853 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.390854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:02.390923 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:02.390927 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:43:02.390981 (+ 54us) client_negotiation.cc:240] Negotiation successful
0927 18:43:02.391031 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":946,"thread_start_us":117,"threads_started":1}
I20260927 18:43:02.391462 31021 tablet_service.cc:1467] Processing CreateTablet for tablet ab81f53adcd24aab85ef215ac0aa8f07 (DEFAULT_TABLE table=test-table [id=bf8b7c5f235046dd90d3ef0b9d24bff7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:02.391458 31152 tablet_service.cc:1467] Processing CreateTablet for tablet ab81f53adcd24aab85ef215ac0aa8f07 (DEFAULT_TABLE table=test-table [id=bf8b7c5f235046dd90d3ef0b9d24bff7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:02.391866 31152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab81f53adcd24aab85ef215ac0aa8f07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:02.391866 31021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab81f53adcd24aab85ef215ac0aa8f07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:02.395576 31359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:02.385667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36002 (local address 127.11.251.195:44811)
0927 18:43:02.386120 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:02.386127 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:02.386142 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:43:02.386189 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:02.386192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:02.386210 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:02.386548 (+ 338us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:02.386614 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.387813 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.389833 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.390664 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.390742 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:02.390849 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:02.395418 (+ 4569us) server_negotiation.cc:299] Negotiation successful
0927 18:43:02.395482 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":125,"threads_started":1}
I20260927 18:43:02.395653 31364 tablet_bootstrap.cc:492] T ab81f53adcd24aab85ef215ac0aa8f07 P ad2a4b5b86fc46e785fd428d348cab88: Bootstrap starting.
I20260927 18:43:02.396598 31364 tablet_bootstrap.cc:654] T ab81f53adcd24aab85ef215ac0aa8f07 P ad2a4b5b86fc46e785fd428d348cab88: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:02.396446 31283 tablet_service.cc:1467] Processing CreateTablet for tablet ab81f53adcd24aab85ef215ac0aa8f07 (DEFAULT_TABLE table=test-table [id=bf8b7c5f235046dd90d3ef0b9d24bff7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:02.396905 31283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab81f53adcd24aab85ef215ac0aa8f07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:02.396984 31364 log.cc:824] T ab81f53adcd24aab85ef215ac0aa8f07 P ad2a4b5b86fc46e785fd428d348cab88: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:02.397421 31363 tablet_bootstrap.cc:492] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee: Bootstrap starting.
I20260927 18:43:02.397943 31364 tablet_bootstrap.cc:492] T ab81f53adcd24aab85ef215ac0aa8f07 P ad2a4b5b86fc46e785fd428d348cab88: No bootstrap required, opened a new log
I20260927 18:43:02.398037 31364 ts_tablet_manager.cc:1397] T ab81f53adcd24aab85ef215ac0aa8f07 P ad2a4b5b86fc46e785fd428d348cab88: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:43:02.398303 31363 tablet_bootstrap.cc:654] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:02.398667 31363 log.cc:824] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:02.399611 31363 tablet_bootstrap.cc:492] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee: No bootstrap required, opened a new log
I20260927 18:43:02.399694 31363 ts_tablet_manager.cc:1397] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:43:02.400463 31364 raft_consensus.cc:348] T ab81f53adcd24aab85ef215ac0aa8f07 P ad2a4b5b86fc46e785fd428d348cab88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713f06dbc2d44e2ca6fb88eeb03c95a3" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44811 } } peers { permanent_uuid: "ad2a4b5b86fc46e785fd428d348cab88" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38357 } } peers { permanent_uuid: "e06b9968322d4cc1ae348d699313dbee" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41959 } }
I20260927 18:43:02.400695 31364 raft_consensus.cc:374] T ab81f53adcd24aab85ef215ac0aa8f07 P ad2a4b5b86fc46e785fd428d348cab88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:02.400746 31364 raft_consensus.cc:729] T ab81f53adcd24aab85ef215ac0aa8f07 P ad2a4b5b86fc46e785fd428d348cab88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2a4b5b86fc46e785fd428d348cab88, State: Initialized, Role: FOLLOWER
I20260927 18:43:02.400866 31364 consensus_queue.cc:260] T ab81f53adcd24aab85ef215ac0aa8f07 P ad2a4b5b86fc46e785fd428d348cab88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713f06dbc2d44e2ca6fb88eeb03c95a3" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44811 } } peers { permanent_uuid: "ad2a4b5b86fc46e785fd428d348cab88" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38357 } } peers { permanent_uuid: "e06b9968322d4cc1ae348d699313dbee" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41959 } }
I20260927 18:43:02.401186 31364 ts_tablet_manager.cc:1428] T ab81f53adcd24aab85ef215ac0aa8f07 P ad2a4b5b86fc46e785fd428d348cab88: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260927 18:43:02.401203 31367 tablet_bootstrap.cc:492] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3: Bootstrap starting.
I20260927 18:43:02.401364 31087 heartbeater.cc:503] Master 127.11.251.254:38349 was elected leader, sending a full tablet report...
I20260927 18:43:02.402025 31367 tablet_bootstrap.cc:654] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:02.402391 31367 log.cc:824] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:02.402885 31363 raft_consensus.cc:348] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713f06dbc2d44e2ca6fb88eeb03c95a3" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44811 } } peers { permanent_uuid: "ad2a4b5b86fc46e785fd428d348cab88" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38357 } } peers { permanent_uuid: "e06b9968322d4cc1ae348d699313dbee" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41959 } }
I20260927 18:43:02.403029 31363 raft_consensus.cc:374] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:02.403052 31363 raft_consensus.cc:729] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e06b9968322d4cc1ae348d699313dbee, State: Initialized, Role: FOLLOWER
I20260927 18:43:02.403148 31367 tablet_bootstrap.cc:492] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3: No bootstrap required, opened a new log
I20260927 18:43:02.403221 31367 ts_tablet_manager.cc:1397] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:43:02.403185 31363 consensus_queue.cc:260] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713f06dbc2d44e2ca6fb88eeb03c95a3" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44811 } } peers { permanent_uuid: "ad2a4b5b86fc46e785fd428d348cab88" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38357 } } peers { permanent_uuid: "e06b9968322d4cc1ae348d699313dbee" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41959 } }
I20260927 18:43:02.403472 31363 ts_tablet_manager.cc:1428] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260927 18:43:02.403538 31218 heartbeater.cc:503] Master 127.11.251.254:38349 was elected leader, sending a full tablet report...
I20260927 18:43:02.405511 31367 raft_consensus.cc:348] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713f06dbc2d44e2ca6fb88eeb03c95a3" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44811 } } peers { permanent_uuid: "ad2a4b5b86fc46e785fd428d348cab88" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38357 } } peers { permanent_uuid: "e06b9968322d4cc1ae348d699313dbee" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41959 } }
I20260927 18:43:02.405740 31367 raft_consensus.cc:374] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:02.405769 31367 raft_consensus.cc:729] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 713f06dbc2d44e2ca6fb88eeb03c95a3, State: Initialized, Role: FOLLOWER
I20260927 18:43:02.405900 31367 consensus_queue.cc:260] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713f06dbc2d44e2ca6fb88eeb03c95a3" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44811 } } peers { permanent_uuid: "ad2a4b5b86fc46e785fd428d348cab88" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38357 } } peers { permanent_uuid: "e06b9968322d4cc1ae348d699313dbee" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41959 } }
I20260927 18:43:02.406194 31367 ts_tablet_manager.cc:1428] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:43:02.406325 31349 heartbeater.cc:503] Master 127.11.251.254:38349 was elected leader, sending a full tablet report...
W20260927 18:43:02.470927 31219 tablet.cc:2369] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:43:02.579485 31088 tablet.cc:2369] T ab81f53adcd24aab85ef215ac0aa8f07 P ad2a4b5b86fc46e785fd428d348cab88: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:43:02.608176 31350 tablet.cc:2369] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:43:02.754957 31370 raft_consensus.cc:482] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:43:02.755203 31370 raft_consensus.cc:504] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713f06dbc2d44e2ca6fb88eeb03c95a3" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44811 } } peers { permanent_uuid: "ad2a4b5b86fc46e785fd428d348cab88" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38357 } } peers { permanent_uuid: "e06b9968322d4cc1ae348d699313dbee" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41959 } }
I20260927 18:43:02.755688 31370 leader_election.cc:288] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 713f06dbc2d44e2ca6fb88eeb03c95a3 (127.11.251.195:44811), ad2a4b5b86fc46e785fd428d348cab88 (127.11.251.193:38357)
I20260927 18:43:02.760632 31359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:02.756297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:55497 (local address 127.11.251.195:44811)
0927 18:43:02.756370 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:02.756372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:02.756381 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:43:02.756415 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:02.756417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:02.756430 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:02.756497 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:02.756775 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.757565 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.759526 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.760126 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.760240 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:02.760249 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:02.760508 (+ 259us) server_negotiation.cc:299] Negotiation successful
0927 18:43:02.760554 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:43:02.760632 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:02.756196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:55153 (local address 127.11.251.193:38357)
0927 18:43:02.756255 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:02.756257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:02.756382 (+ 125us) server_negotiation.cc:407] Connection header received
0927 18:43:02.756466 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:02.756467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:02.756480 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:02.756540 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:02.756775 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.757613 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.759526 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.760163 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.760240 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:02.760249 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:02.760508 (+ 259us) server_negotiation.cc:299] Negotiation successful
0927 18:43:02.760554 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 18:43:02.760677 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:02.755903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:44811 (local address 127.11.251.194:55497)
0927 18:43:02.756055 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:02.756068 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:02.756179 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:02.756546 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:02.756547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:02.756550 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:02.756673 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:02.756675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.757736 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.757737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:02.759366 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:02.759369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.760289 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.760290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:02.760378 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:02.760382 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:43:02.760455 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 18:43:02.760522 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":163,"thread_start_us":104,"threads_started":1}
I20260927 18:43:02.760677 31373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:02.756093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:38357 (local address 127.11.251.194:55153)
0927 18:43:02.756276 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:02.756291 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:02.756388 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:02.756576 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:02.756577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:02.756579 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:02.756672 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:02.756675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.757735 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.757737 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:02.759366 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:02.759369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.760288 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.760290 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:02.760371 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:02.760378 (+ 7us) client_negotiation.cc:769] Sending connection context
0927 18:43:02.760455 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 18:43:02.760523 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":46,"thread_start_us":119,"threads_started":1}
I20260927 18:43:02.761456 31303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab81f53adcd24aab85ef215ac0aa8f07" candidate_uuid: "e06b9968322d4cc1ae348d699313dbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "713f06dbc2d44e2ca6fb88eeb03c95a3" is_pre_election: true
I20260927 18:43:02.761613 31041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab81f53adcd24aab85ef215ac0aa8f07" candidate_uuid: "e06b9968322d4cc1ae348d699313dbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2a4b5b86fc46e785fd428d348cab88" is_pre_election: true
I20260927 18:43:02.761797 31303 raft_consensus.cc:2454] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e06b9968322d4cc1ae348d699313dbee in term 0.
I20260927 18:43:02.761893 31041 raft_consensus.cc:2454] T ab81f53adcd24aab85ef215ac0aa8f07 P ad2a4b5b86fc46e785fd428d348cab88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e06b9968322d4cc1ae348d699313dbee in term 0.
I20260927 18:43:02.762159 31105 leader_election.cc:302] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 713f06dbc2d44e2ca6fb88eeb03c95a3, e06b9968322d4cc1ae348d699313dbee; no voters:
I20260927 18:43:02.762332 31370 raft_consensus.cc:2781] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:43:02.762391 31370 raft_consensus.cc:482] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:43:02.762422 31370 raft_consensus.cc:3037] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:02.763295 31370 raft_consensus.cc:504] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713f06dbc2d44e2ca6fb88eeb03c95a3" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44811 } } peers { permanent_uuid: "ad2a4b5b86fc46e785fd428d348cab88" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38357 } } peers { permanent_uuid: "e06b9968322d4cc1ae348d699313dbee" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41959 } }
I20260927 18:43:02.763484 31370 leader_election.cc:288] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee [CANDIDATE]: Term 1 election: Requested vote from peers 713f06dbc2d44e2ca6fb88eeb03c95a3 (127.11.251.195:44811), ad2a4b5b86fc46e785fd428d348cab88 (127.11.251.193:38357)
I20260927 18:43:02.763689 31303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab81f53adcd24aab85ef215ac0aa8f07" candidate_uuid: "e06b9968322d4cc1ae348d699313dbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "713f06dbc2d44e2ca6fb88eeb03c95a3"
I20260927 18:43:02.763710 31041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab81f53adcd24aab85ef215ac0aa8f07" candidate_uuid: "e06b9968322d4cc1ae348d699313dbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2a4b5b86fc46e785fd428d348cab88"
I20260927 18:43:02.763806 31041 raft_consensus.cc:3037] T ab81f53adcd24aab85ef215ac0aa8f07 P ad2a4b5b86fc46e785fd428d348cab88 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:02.763810 31303 raft_consensus.cc:3037] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:02.765040 31041 raft_consensus.cc:2454] T ab81f53adcd24aab85ef215ac0aa8f07 P ad2a4b5b86fc46e785fd428d348cab88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e06b9968322d4cc1ae348d699313dbee in term 1.
I20260927 18:43:02.765040 31303 raft_consensus.cc:2454] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e06b9968322d4cc1ae348d699313dbee in term 1.
I20260927 18:43:02.765324 31106 leader_election.cc:302] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2a4b5b86fc46e785fd428d348cab88, e06b9968322d4cc1ae348d699313dbee; no voters:
I20260927 18:43:02.765498 31370 raft_consensus.cc:2781] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:02.765758 31370 raft_consensus.cc:686] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee [term 1 LEADER]: Becoming Leader. State: Replica: e06b9968322d4cc1ae348d699313dbee, State: Running, Role: LEADER
I20260927 18:43:02.765882 31370 consensus_queue.cc:237] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713f06dbc2d44e2ca6fb88eeb03c95a3" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44811 } } peers { permanent_uuid: "ad2a4b5b86fc46e785fd428d348cab88" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38357 } } peers { permanent_uuid: "e06b9968322d4cc1ae348d699313dbee" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41959 } }
I20260927 18:43:02.766958 30898 catalog_manager.cc:5495] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee reported cstate change: term changed from 0 to 1, leader changed from <none> to e06b9968322d4cc1ae348d699313dbee (127.11.251.194). New cstate: current_term: 1 leader_uuid: "e06b9968322d4cc1ae348d699313dbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713f06dbc2d44e2ca6fb88eeb03c95a3" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad2a4b5b86fc46e785fd428d348cab88" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e06b9968322d4cc1ae348d699313dbee" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 41959 } health_report { overall_health: HEALTHY } } }
I20260927 18:43:02.836206 31361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:02.833240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54876 (local address 127.11.251.194:41959)
0927 18:43:02.833349 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:02.833351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:02.833365 (+ 14us) server_negotiation.cc:407] Connection header received
0927 18:43:02.833422 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:02.833423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:02.833454 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:02.833556 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:02.833770 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.834330 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.835292 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:02.835510 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:02.835608 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:02.835808 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:02.835916 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:02.836051 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:02.836086 (+ 35us) server_negotiation.cc:299] Negotiation successful
0927 18:43:02.836099 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 18:43:02.838826 31303 raft_consensus.cc:1261] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3 [term 1 FOLLOWER]: Refusing update from remote peer e06b9968322d4cc1ae348d699313dbee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:02.838936 31041 raft_consensus.cc:1261] T ab81f53adcd24aab85ef215ac0aa8f07 P ad2a4b5b86fc46e785fd428d348cab88 [term 1 FOLLOWER]: Refusing update from remote peer e06b9968322d4cc1ae348d699313dbee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:02.839345 31374 consensus_queue.cc:1035] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "713f06dbc2d44e2ca6fb88eeb03c95a3" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 44811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:02.839563 31370 consensus_queue.cc:1035] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2a4b5b86fc46e785fd428d348cab88" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 38357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:02.842703 31377 mvcc.cc:205] Tried to move back new op lower bound from 7334029651304148992 to 7334029651009744896. Current Snapshot: MvccSnapshot[applied={T|T < 7334029651304148992}]
I20260927 18:43:02.843035 31379 mvcc.cc:205] Tried to move back new op lower bound from 7334029651304148992 to 7334029651009744896. Current Snapshot: MvccSnapshot[applied={T|T < 7334029651304148992 or (T in {7334029651304148992})}]
I20260927 18:43:02.845374 30897 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:43:02.845481 30897 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:43:02.846493 30897 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 57131633effa49b68ef0667b2ae09a9f: Sending DeleteTablet for 3 replicas of tablet ab81f53adcd24aab85ef215ac0aa8f07
I20260927 18:43:02.846825 31283 tablet_service.cc:1514] Processing DeleteTablet for tablet ab81f53adcd24aab85ef215ac0aa8f07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:02 UTC) from {username='slave'} at 127.0.0.1:36002
I20260927 18:43:02.846858 31021 tablet_service.cc:1514] Processing DeleteTablet for tablet ab81f53adcd24aab85ef215ac0aa8f07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:02 UTC) from {username='slave'} at 127.0.0.1:35004
I20260927 18:43:02.846915 31152 tablet_service.cc:1514] Processing DeleteTablet for tablet ab81f53adcd24aab85ef215ac0aa8f07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:02 UTC) from {username='slave'} at 127.0.0.1:54864
I20260927 18:43:02.847142 31388 tablet_replica.cc:330] T ab81f53adcd24aab85ef215ac0aa8f07 P ad2a4b5b86fc46e785fd428d348cab88: stopping tablet replica
I20260927 18:43:02.847275 31388 raft_consensus.cc:2229] T ab81f53adcd24aab85ef215ac0aa8f07 P ad2a4b5b86fc46e785fd428d348cab88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:43:02.847430 31388 raft_consensus.cc:2258] T ab81f53adcd24aab85ef215ac0aa8f07 P ad2a4b5b86fc46e785fd428d348cab88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:02.847683 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 30959
I20260927 18:43:02.847746 31389 tablet_replica.cc:330] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee: stopping tablet replica
I20260927 18:43:02.847831 31389 raft_consensus.cc:2229] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:43:02.847975 31389 raft_consensus.cc:2258] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:02.848703 31389 ts_tablet_manager.cc:1905] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:02.851158 31389 ts_tablet_manager.cc:1918] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:43:02.851245 31389 log.cc:1196] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/ab81f53adcd24aab85ef215ac0aa8f07
I20260927 18:43:02.851593 31389 ts_tablet_manager.cc:1939] T ab81f53adcd24aab85ef215ac0aa8f07 P e06b9968322d4cc1ae348d699313dbee: Deleting consensus metadata
I20260927 18:43:02.852116 30884 catalog_manager.cc:4841] TS e06b9968322d4cc1ae348d699313dbee (127.11.251.194:41959): tablet ab81f53adcd24aab85ef215ac0aa8f07 (table test-table [id=bf8b7c5f235046dd90d3ef0b9d24bff7]) successfully deleted
I20260927 18:43:02.847317 31387 tablet_replica.cc:330] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3: stopping tablet replica
W20260927 18:43:02.856631 30884 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.251.193:38357 (error 108)
I20260927 18:43:02.856808 31387 raft_consensus.cc:2229] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:43:02.856967 31387 raft_consensus.cc:2258] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:02.857398 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:02.857239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:38357 (local address 127.0.0.1:35014)
0927 18:43:02.857293 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:02.857348 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.193:38357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260927 18:43:02.857512 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 31090
I20260927 18:43:02.857563 31387 ts_tablet_manager.cc:1905] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 18:43:02.857738 30884 catalog_manager.cc:4584] TS ad2a4b5b86fc46e785fd428d348cab88 (127.11.251.193:38357): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ab81f53adcd24aab85ef215ac0aa8f07: Network error: Client connection negotiation failed: client connection to 127.11.251.193:38357: connect: Connection refused (error 111)
I20260927 18:43:02.859735 31387 ts_tablet_manager.cc:1918] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:43:02.859815 31387 log.cc:1196] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/ab81f53adcd24aab85ef215ac0aa8f07
I20260927 18:43:02.860075 31387 ts_tablet_manager.cc:1939] T ab81f53adcd24aab85ef215ac0aa8f07 P 713f06dbc2d44e2ca6fb88eeb03c95a3: Deleting consensus metadata
I20260927 18:43:02.860646 30883 catalog_manager.cc:4841] TS 713f06dbc2d44e2ca6fb88eeb03c95a3 (127.11.251.195:44811): tablet ab81f53adcd24aab85ef215ac0aa8f07 (table test-table [id=bf8b7c5f235046dd90d3ef0b9d24bff7]) successfully deleted
I20260927 18:43:02.863690 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 31221
I20260927 18:43:02.869328 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 30868
2026-09-27T18:43:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1139 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:02 dist-test-slave-z47j krb5kdc[31393](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:02 dist-test-slave-z47j krb5kdc[31393](info): set up 2 sockets
Sep 27 18:43:02 dist-test-slave-z47j krb5kdc[31393](info): commencing operation
krb5kdc: starting...
W20260927 18:43:03.925763 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:04 dist-test-slave-z47j krb5kdc[31393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:43:04.062042 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:35675
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:34129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:35675
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:04.146752 31406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:04.147028 31406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:04.147068 31406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:04.148216 31406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:04.148299 31406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:04.148324 31406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:04.148345 31406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:04.148403 31406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:04.149992 31406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:34129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:35675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:35675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:04.150297 31406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:04.150540 31406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:04.156065 31414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:04.156103 31412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:04.156433 31406 server_base.cc:1034] running on GCE node
W20260927 18:43:04.156103 31411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:04.156845 31406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:04.157143 31406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:04.158367 31406 hybrid_clock.cc:648] HybridClock initialized: now 1790534584158349 us; error 53 us; skew 500 ppm
Sep 27 18:43:04 dist-test-slave-z47j krb5kdc[31393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534584, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:04.161291 31406 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:43:04.161899 31406 webserver.cc:460] Webserver started at http://127.11.251.254:36729/ using document root <none> and password file <none>
I20260927 18:43:04.162178 31406 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:04.162254 31406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:04.162424 31406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:04.163610 31406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "6729314ff6484aa882582b25fe7ce8ef"
format_stamp: "Formatted at 2026-09-27 18:43:04 on dist-test-slave-z47j"
server_key: "54fb2c65e6624110a04ba2df76d8e628"
server_key_iv: "26328a79aa610ce3390379e416974533"
server_key_version: "encryptionkey@0"
I20260927 18:43:04.164004 31406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "6729314ff6484aa882582b25fe7ce8ef"
format_stamp: "Formatted at 2026-09-27 18:43:04 on dist-test-slave-z47j"
server_key: "54fb2c65e6624110a04ba2df76d8e628"
server_key_iv: "26328a79aa610ce3390379e416974533"
server_key_version: "encryptionkey@0"
I20260927 18:43:04.165637 31406 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:43:04.166680 31421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:04.167016 31406 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:43:04.167196 31406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "6729314ff6484aa882582b25fe7ce8ef"
format_stamp: "Formatted at 2026-09-27 18:43:04 on dist-test-slave-z47j"
server_key: "54fb2c65e6624110a04ba2df76d8e628"
server_key_iv: "26328a79aa610ce3390379e416974533"
server_key_version: "encryptionkey@0"
I20260927 18:43:04.167285 31406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:04.178589 31406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:04.180899 31406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:04.181066 31406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:04.185856 31406 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:35675
I20260927 18:43:04.185967 31473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:35675 every 8 connection(s)
I20260927 18:43:04.186311 31406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:04.187390 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:04.188057 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 31406
I20260927 18:43:04.188203 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:04.188485 12271 external_mini_cluster.cc:1428] Setting key 7ed1064fcc486b3a8a6188f55cf2cc02
I20260927 18:43:04.190996 31474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef: Bootstrap starting.
I20260927 18:43:04.191825 31474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:04.192169 31474 log.cc:824] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:04.193085 31474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef: No bootstrap required, opened a new log
Sep 27 18:43:04 dist-test-slave-z47j krb5kdc[31393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:43:04.195806 31474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6729314ff6484aa882582b25fe7ce8ef" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 35675 } }
I20260927 18:43:04.196030 31474 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:04.196063 31474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6729314ff6484aa882582b25fe7ce8ef, State: Initialized, Role: FOLLOWER
I20260927 18:43:04.196231 31474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6729314ff6484aa882582b25fe7ce8ef" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 35675 } }
I20260927 18:43:04.196326 31474 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:04.196353 31474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:04.196435 31474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:04.197176 31474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6729314ff6484aa882582b25fe7ce8ef" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 35675 } }
I20260927 18:43:04.197319 31474 leader_election.cc:302] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6729314ff6484aa882582b25fe7ce8ef; no voters:
I20260927 18:43:04.197623 31474 leader_election.cc:288] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:04.197692 31479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:04.197934 31479 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef [term 1 LEADER]: Becoming Leader. State: Replica: 6729314ff6484aa882582b25fe7ce8ef, State: Running, Role: LEADER
I20260927 18:43:04.198009 31474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:04.198084 31479 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6729314ff6484aa882582b25fe7ce8ef" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 35675 } }
I20260927 18:43:04.198556 31479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6729314ff6484aa882582b25fe7ce8ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6729314ff6484aa882582b25fe7ce8ef" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 35675 } } }
I20260927 18:43:04.198662 31479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:04.198836 31479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6729314ff6484aa882582b25fe7ce8ef. Latest consensus state: current_term: 1 leader_uuid: "6729314ff6484aa882582b25fe7ce8ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6729314ff6484aa882582b25fe7ce8ef" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 35675 } } }
I20260927 18:43:04.198906 31479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:04.199052 31486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:04.199709 31486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:04.201368 31477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:04.189085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58384 (local address 127.11.251.254:35675)
0927 18:43:04.189333 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:04.189342 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:04.189370 (+ 28us) server_negotiation.cc:407] Connection header received
0927 18:43:04.190086 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:04.190093 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:04.190150 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:04.190302 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:04.190949 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.192026 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.192975 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.193455 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.193614 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:04.196553 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:04.196557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:04.196557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:04.196568 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:04.199397 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:04.199893 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:04.199897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:04.199898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:04.200010 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:04.200315 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:04.200316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:04.200317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:04.200668 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:04.200864 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:04.200943 (+ 79us) server_negotiation.cc:299] Negotiation successful
0927 18:43:04.201106 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":118,"threads_started":1}
I20260927 18:43:04.202083 31486 catalog_manager.cc:1347] Generated new cluster ID: 742dc257429947b8b26b57210c8421d1
I20260927 18:43:04.202214 31486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:04.210098 31486 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:04.210691 31486 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:04.215384 31486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef: Generated new TSK 0
I20260927 18:43:04.215559 31486 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.251.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
Entry for principal kudu/127.11.251.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab.
I20260927 18:43:04.246407 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:35675
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:34129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:04.328089 31500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:04.328325 31500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:04.328362 31500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:04.329473 31500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:04.329542 31500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:04.329591 31500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:43:04.331084 31500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:34129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.193.keytab
--principal=kudu/127.11.251.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:35675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:04.331344 31500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:04.331579 31500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:04.337671 31508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:04.337693 31506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:04.337751 31505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:04.337828 31500 server_base.cc:1034] running on GCE node
I20260927 18:43:04.338260 31500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:04.338582 31500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:04.339761 31500 hybrid_clock.cc:648] HybridClock initialized: now 1790534584339731 us; error 54 us; skew 500 ppm
Sep 27 18:43:04 dist-test-slave-z47j krb5kdc[31393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534584, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:04.342669 31500 init.cc:377] Logged in from keytab as kudu/127.11.251.193@KRBTEST.COM (short username kudu)
I20260927 18:43:04.343140 31500 webserver.cc:460] Webserver started at http://127.11.251.193:44401/ using document root <none> and password file <none>
I20260927 18:43:04.343369 31500 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:04.343417 31500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:04.343526 31500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:04.344667 31500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "4c3240babd454d09b1bd923aec0ea162"
format_stamp: "Formatted at 2026-09-27 18:43:04 on dist-test-slave-z47j"
server_key: "2a806620e2e5f802c4e5938d59a62ffb"
server_key_iv: "e900b9ec5b5380282014b8b61cf7e4f8"
server_key_version: "encryptionkey@0"
I20260927 18:43:04.345000 31500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "4c3240babd454d09b1bd923aec0ea162"
format_stamp: "Formatted at 2026-09-27 18:43:04 on dist-test-slave-z47j"
server_key: "2a806620e2e5f802c4e5938d59a62ffb"
server_key_iv: "e900b9ec5b5380282014b8b61cf7e4f8"
server_key_version: "encryptionkey@0"
I20260927 18:43:04.346643 31500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:43:04.347467 31515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:04.347715 31500 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 18:43:04.347823 31500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "4c3240babd454d09b1bd923aec0ea162"
format_stamp: "Formatted at 2026-09-27 18:43:04 on dist-test-slave-z47j"
server_key: "2a806620e2e5f802c4e5938d59a62ffb"
server_key_iv: "e900b9ec5b5380282014b8b61cf7e4f8"
server_key_version: "encryptionkey@0"
I20260927 18:43:04.347913 31500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:04.359210 31500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:04.361550 31500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:04.361737 31500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:04.362023 31500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:04.362375 31500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:04.362434 31500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:04.362469 31500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:04.362502 31500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:04.369112 31500 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:43003
I20260927 18:43:04.369230 31628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:43003 every 8 connection(s)
I20260927 18:43:04.369549 31500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:43:04.372884 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 31500
I20260927 18:43:04.373023 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:43:04.373281 12271 external_mini_cluster.cc:1428] Setting key 00aa4c0ac8cfd228eecfb9a7738c05d1
Sep 27 18:43:04 dist-test-slave-z47j krb5kdc[31393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534584, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.193@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:43:04.382103 31477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:04.370562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:51721 (local address 127.11.251.254:35675)
0927 18:43:04.370674 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:04.370677 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:04.370879 (+ 202us) server_negotiation.cc:407] Connection header received
0927 18:43:04.371623 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:04.371624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:04.371638 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:04.371696 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:04.373023 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.373854 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.375847 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.376168 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.376272 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:04.379118 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:04.379119 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:04.379120 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:04.379127 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:04.380634 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:04.381078 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:04.381079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:04.381080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:04.381146 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:04.381401 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:04.381402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:04.381403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:04.381526 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:04.381625 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:04.381912 (+ 287us) server_negotiation.cc:299] Negotiation successful
0927 18:43:04.382001 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260927 18:43:04.382376 31631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:04.370445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:35675 (local address 127.11.251.193:51721)
0927 18:43:04.370706 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:04.370762 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:04.371502 (+ 740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:04.371797 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:04.371802 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:04.372499 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:04.372832 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:04.372839 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.373992 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.373993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:04.375726 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:04.375730 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.376300 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.376301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:04.376463 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:04.377067 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:04.377076 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:04.379002 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:04.380743 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:04.380750 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:04.380752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:04.380997 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:04.381217 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:04.381217 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:04.381218 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:04.381331 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:04.381621 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:04.381628 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:04.381777 (+ 149us) client_negotiation.cc:769] Sending connection context
0927 18:43:04.381890 (+ 113us) client_negotiation.cc:240] Negotiation successful
0927 18:43:04.382071 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":120,"threads_started":1}
I20260927 18:43:04.382892 31629 heartbeater.cc:347] Connected to a master server at 127.11.251.254:35675
I20260927 18:43:04.383054 31629 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:04.383343 31629 heartbeater.cc:511] Master 127.11.251.254:35675 requested a full tablet report, sending...
I20260927 18:43:04.384073 31438 ts_manager.cc:194] Registered new tserver with Master: 4c3240babd454d09b1bd923aec0ea162 (127.11.251.193:43003)
WARNING: no policy specified for kudu/127.11.251.194@KRBTEST.COM; defaulting to no policy
I20260927 18:43:04.384819 31438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.193@KRBTEST.COM'} at 127.11.251.193:51721
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
Entry for principal kudu/127.11.251.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab.
I20260927 18:43:04.406155 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:35675
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:34129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:04.502377 31634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:04.502637 31634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:04.502662 31634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:04.503829 31634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:04.503868 31634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:04.503894 31634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:43:04.505338 31634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:34129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.194.keytab
--principal=kudu/127.11.251.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:35675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:04.505565 31634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:04.505831 31634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:04.511392 31639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:04.511375 31640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:04.511373 31642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:04.511854 31634 server_base.cc:1034] running on GCE node
I20260927 18:43:04.512071 31634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:04.512285 31634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:04.513489 31634 hybrid_clock.cc:648] HybridClock initialized: now 1790534584513468 us; error 41 us; skew 500 ppm
Sep 27 18:43:04 dist-test-slave-z47j krb5kdc[31393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534584, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:04.516217 31634 init.cc:377] Logged in from keytab as kudu/127.11.251.194@KRBTEST.COM (short username kudu)
I20260927 18:43:04.516805 31634 webserver.cc:460] Webserver started at http://127.11.251.194:43795/ using document root <none> and password file <none>
I20260927 18:43:04.517055 31634 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:04.517122 31634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:04.517299 31634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:04.518563 31634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "d8cbd20d78554521940ebfcecd87bbd2"
format_stamp: "Formatted at 2026-09-27 18:43:04 on dist-test-slave-z47j"
server_key: "af86ad1d8aa2c39a646ebbcf795b25b7"
server_key_iv: "6506ec9d40ef6ca204443a3d17da6cd3"
server_key_version: "encryptionkey@0"
I20260927 18:43:04.518960 31634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "d8cbd20d78554521940ebfcecd87bbd2"
format_stamp: "Formatted at 2026-09-27 18:43:04 on dist-test-slave-z47j"
server_key: "af86ad1d8aa2c39a646ebbcf795b25b7"
server_key_iv: "6506ec9d40ef6ca204443a3d17da6cd3"
server_key_version: "encryptionkey@0"
I20260927 18:43:04.520504 31634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 18:43:04.521409 31649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:04.521754 31634 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:04.521880 31634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "d8cbd20d78554521940ebfcecd87bbd2"
format_stamp: "Formatted at 2026-09-27 18:43:04 on dist-test-slave-z47j"
server_key: "af86ad1d8aa2c39a646ebbcf795b25b7"
server_key_iv: "6506ec9d40ef6ca204443a3d17da6cd3"
server_key_version: "encryptionkey@0"
I20260927 18:43:04.522020 31634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:04.535651 31634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:04.538106 31634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:04.538234 31634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:04.538494 31634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:04.538833 31634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:04.538875 31634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:04.538900 31634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:04.538914 31634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:04.546555 31634 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:32921
I20260927 18:43:04.546685 31762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:32921 every 8 connection(s)
I20260927 18:43:04.547060 31634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:43:04.552740 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 31634
I20260927 18:43:04.552969 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:43:04.553267 12271 external_mini_cluster.cc:1428] Setting key 85ac8737a088e9b04e4491e553710f9d
Sep 27 18:43:04 dist-test-slave-z47j krb5kdc[31393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534584, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.194@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:43:04.560312 31477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:04.548063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:35659 (local address 127.11.251.254:35675)
0927 18:43:04.549458 (+ 1395us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:04.549461 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:04.549472 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:43:04.549505 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:04.549507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:04.549530 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:04.549647 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:04.550820 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.551402 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.552760 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.553086 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.553140 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:04.556632 (+ 3492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:04.556635 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:04.556635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:04.556649 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:04.558388 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:04.559046 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:04.559047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:04.559047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:04.559106 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:04.559467 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:04.559468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:04.559469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:04.559626 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:04.559718 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:04.560155 (+ 437us) server_negotiation.cc:299] Negotiation successful
0927 18:43:04.560242 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260927 18:43:04.560702 31765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:04.547980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:35675 (local address 127.11.251.194:35659)
0927 18:43:04.548262 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:04.548325 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:04.549272 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:04.549744 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:04.549749 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:04.550433 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:04.550685 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:04.550689 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.551509 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.551510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:04.552644 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:04.552647 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.553257 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.553258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:04.553421 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:04.554150 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:04.554162 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:04.556444 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:04.558522 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:04.558531 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:04.558534 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:04.558943 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:04.559187 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:04.559188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:04.559189 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:04.559359 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:04.559726 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:04.559735 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:04.559963 (+ 228us) client_negotiation.cc:769] Sending connection context
0927 18:43:04.560126 (+ 163us) client_negotiation.cc:240] Negotiation successful
0927 18:43:04.560369 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":109,"threads_started":1}
I20260927 18:43:04.561514 31763 heartbeater.cc:347] Connected to a master server at 127.11.251.254:35675
I20260927 18:43:04.561694 31763 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:04.561949 31763 heartbeater.cc:511] Master 127.11.251.254:35675 requested a full tablet report, sending...
I20260927 18:43:04.562460 31438 ts_manager.cc:194] Registered new tserver with Master: d8cbd20d78554521940ebfcecd87bbd2 (127.11.251.194:32921)
I20260927 18:43:04.563117 31438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.194@KRBTEST.COM'} at 127.11.251.194:35659
WARNING: no policy specified for kudu/127.11.251.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
Entry for principal kudu/127.11.251.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab.
I20260927 18:43:04.585297 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:35675
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:34129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:04.675642 31768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:04.675879 31768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:04.675906 31768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:04.677081 31768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:04.677124 31768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:04.677150 31768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:43:04.678699 31768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:34129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.195.keytab
--principal=kudu/127.11.251.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:35675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:04.678915 31768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:04.679138 31768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:04.684701 31773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:04.684703 31776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:04.684792 31774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:04.684831 31768 server_base.cc:1034] running on GCE node
I20260927 18:43:04.685225 31768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:04.685493 31768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:04.686673 31768 hybrid_clock.cc:648] HybridClock initialized: now 1790534584686662 us; error 30 us; skew 500 ppm
Sep 27 18:43:04 dist-test-slave-z47j krb5kdc[31393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534584, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:04.689534 31768 init.cc:377] Logged in from keytab as kudu/127.11.251.195@KRBTEST.COM (short username kudu)
I20260927 18:43:04.690141 31768 webserver.cc:460] Webserver started at http://127.11.251.195:33803/ using document root <none> and password file <none>
I20260927 18:43:04.690407 31768 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:04.690485 31768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:04.690646 31768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:04.691864 31768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "273b623cdea64c319e0d540e935ba261"
format_stamp: "Formatted at 2026-09-27 18:43:04 on dist-test-slave-z47j"
server_key: "2d5d88f2cbe54f8587f8dcd3ed09d514"
server_key_iv: "e333e5c3967984b8f1668d42c3bdb15c"
server_key_version: "encryptionkey@0"
I20260927 18:43:04.692292 31768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "273b623cdea64c319e0d540e935ba261"
format_stamp: "Formatted at 2026-09-27 18:43:04 on dist-test-slave-z47j"
server_key: "2d5d88f2cbe54f8587f8dcd3ed09d514"
server_key_iv: "e333e5c3967984b8f1668d42c3bdb15c"
server_key_version: "encryptionkey@0"
I20260927 18:43:04.693847 31768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260927 18:43:04.694669 31783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:04.694882 31768 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:04.694994 31768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "273b623cdea64c319e0d540e935ba261"
format_stamp: "Formatted at 2026-09-27 18:43:04 on dist-test-slave-z47j"
server_key: "2d5d88f2cbe54f8587f8dcd3ed09d514"
server_key_iv: "e333e5c3967984b8f1668d42c3bdb15c"
server_key_version: "encryptionkey@0"
I20260927 18:43:04.695071 31768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:04.734767 31768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:04.737150 31768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:04.737332 31768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:04.737720 31768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:04.738114 31768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:04.738178 31768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:04.738230 31768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:04.738260 31768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:04.746213 31768 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:45557
I20260927 18:43:04.746316 31896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:45557 every 8 connection(s)
I20260927 18:43:04.746731 31768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:43:04.751362 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 31768
I20260927 18:43:04.751519 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:43:04.751729 12271 external_mini_cluster.cc:1428] Setting key 0777a2d8e1cf65afadd2f6f9c723ff3e
Sep 27 18:43:04 dist-test-slave-z47j krb5kdc[31393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534584, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.195@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:43:04.758283 31477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:04.747732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:45005 (local address 127.11.251.254:35675)
0927 18:43:04.747814 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:04.747816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:04.748056 (+ 240us) server_negotiation.cc:407] Connection header received
0927 18:43:04.748654 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:04.748656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:04.748667 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:04.748751 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:04.749964 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.750667 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.752148 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.752455 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.752533 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:04.755193 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:04.755194 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:04.755195 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:04.755202 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:04.756722 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:04.757214 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:04.757215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:04.757216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:04.757278 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:04.757562 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:04.757562 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:04.757563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:04.757781 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:04.757853 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:04.758136 (+ 283us) server_negotiation.cc:299] Negotiation successful
0927 18:43:04.758214 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 18:43:04.758540 31899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:04.747701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:35675 (local address 127.11.251.195:45005)
0927 18:43:04.747890 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:04.747942 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:04.748556 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:04.748820 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:04.748825 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:04.749514 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:04.749841 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:04.749848 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.750774 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.750775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:04.752040 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:04.752044 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.752554 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.752555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:04.752714 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:04.753376 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:04.753385 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:04.755097 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:04.756818 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:04.756825 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:04.756827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:04.757089 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:04.757382 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:04.757383 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:04.757383 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:04.757493 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:04.757859 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:04.757864 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:04.758000 (+ 136us) client_negotiation.cc:769] Sending connection context
0927 18:43:04.758099 (+ 99us) client_negotiation.cc:240] Negotiation successful
0927 18:43:04.758239 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":101,"threads_started":1}
I20260927 18:43:04.759230 31897 heartbeater.cc:347] Connected to a master server at 127.11.251.254:35675
I20260927 18:43:04.759351 31897 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:04.759562 31897 heartbeater.cc:511] Master 127.11.251.254:35675 requested a full tablet report, sending...
I20260927 18:43:04.760010 31438 ts_manager.cc:194] Registered new tserver with Master: 273b623cdea64c319e0d540e935ba261 (127.11.251.195:45557)
I20260927 18:43:04.760452 31438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.251.195@KRBTEST.COM'} at 127.11.251.195:45005
I20260927 18:43:04.764108 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:43:04.772250 31477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:04.765746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58396 (local address 127.11.251.254:35675)
0927 18:43:04.765844 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:04.765846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:04.766021 (+ 175us) server_negotiation.cc:407] Connection header received
0927 18:43:04.766073 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:04.766074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:04.766084 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:04.766143 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:04.766737 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.767262 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.768011 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.768212 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.768282 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:04.769481 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:04.769482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:04.769483 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:04.769488 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:04.771063 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:04.771426 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:04.771427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:04.771427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:04.771486 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:04.771679 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:04.771680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:04.771680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:04.771860 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:04.771955 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:04.772055 (+ 100us) server_negotiation.cc:299] Negotiation successful
0927 18:43:04.772123 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 18:43:04.774124 31438 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:43:04.776108 31438 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:43:04.783655 31912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:04.779444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38434 (local address 127.11.251.193:43003)
0927 18:43:04.779671 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:04.779677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:04.779689 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:43:04.779772 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:04.779774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:04.779798 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:04.780085 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:04.780169 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.780940 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.782440 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.783159 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.783253 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:04.783315 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:04.783499 (+ 184us) server_negotiation.cc:299] Negotiation successful
0927 18:43:04.783549 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":95,"threads_started":1}
I20260927 18:43:04.783639 31908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:04.779205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:43003 (local address 127.0.0.1:38434)
0927 18:43:04.779616 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:04.779630 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:04.779695 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:04.779911 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:04.779915 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:04.779918 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:04.779999 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:04.780002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.781054 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.781055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:04.782327 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:04.782329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.783251 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.783252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:04.783359 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:04.783375 (+ 16us) client_negotiation.cc:769] Sending connection context
0927 18:43:04.783480 (+ 105us) client_negotiation.cc:240] Negotiation successful
0927 18:43:04.783553 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":115,"threads_started":1}
I20260927 18:43:04.783949 31909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:04.779204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:45557 (local address 127.0.0.1:41514)
0927 18:43:04.779431 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:04.779486 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:04.779564 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:04.780053 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:04.780054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:04.780056 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:04.780186 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:04.780188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.781540 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.781541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:04.782624 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:04.782626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.783668 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.783670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:04.783746 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:04.783752 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:43:04.783801 (+ 49us) client_negotiation.cc:240] Negotiation successful
0927 18:43:04.783864 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":100,"thread_start_us":74,"threads_started":1}
I20260927 18:43:04.784086 31910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:04.779207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:32921 (local address 127.0.0.1:59390)
0927 18:43:04.779698 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:04.779708 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:04.779765 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:04.780043 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:04.780045 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:04.780048 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:04.780148 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:04.780151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.781568 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.781569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:04.782821 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:04.782824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.783862 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.783862 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:04.783924 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:04.783927 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:43:04.783973 (+ 46us) client_negotiation.cc:240] Negotiation successful
0927 18:43:04.784022 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":3,"thread_start_us":107,"threads_started":1}
I20260927 18:43:04.784181 31911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:04.779409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41514 (local address 127.11.251.195:45557)
0927 18:43:04.779834 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:04.779839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:04.779849 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:43:04.779890 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:04.779892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:04.779907 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:04.779981 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:04.780264 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.781424 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.782720 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.783489 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.783571 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:04.783626 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:04.784025 (+ 399us) server_negotiation.cc:299] Negotiation successful
0927 18:43:04.784083 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":113,"threads_started":1}
I20260927 18:43:04.784337 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:04.779515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59390 (local address 127.11.251.194:32921)
0927 18:43:04.779706 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:04.779712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:04.779823 (+ 111us) server_negotiation.cc:407] Connection header received
0927 18:43:04.779871 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:04.779873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:04.779890 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:04.779964 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:04.780238 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.781418 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.782935 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.783692 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.783763 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:04.783812 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:04.784188 (+ 376us) server_negotiation.cc:299] Negotiation successful
0927 18:43:04.784244 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":82,"threads_started":1}
I20260927 18:43:04.784852 31563 tablet_service.cc:1467] Processing CreateTablet for tablet a2976e0cb2ab4d7980ec185f2cfba170 (DEFAULT_TABLE table=test-table [id=b286634779c94045bba024eca57a6add]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:04.784929 31697 tablet_service.cc:1467] Processing CreateTablet for tablet a2976e0cb2ab4d7980ec185f2cfba170 (DEFAULT_TABLE table=test-table [id=b286634779c94045bba024eca57a6add]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:04.784926 31831 tablet_service.cc:1467] Processing CreateTablet for tablet a2976e0cb2ab4d7980ec185f2cfba170 (DEFAULT_TABLE table=test-table [id=b286634779c94045bba024eca57a6add]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:04.785161 31563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2976e0cb2ab4d7980ec185f2cfba170. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:04.785305 31697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2976e0cb2ab4d7980ec185f2cfba170. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:04.785305 31831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2976e0cb2ab4d7980ec185f2cfba170. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:04.788123 31914 tablet_bootstrap.cc:492] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162: Bootstrap starting.
I20260927 18:43:04.788841 31914 tablet_bootstrap.cc:654] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:04.788945 31915 tablet_bootstrap.cc:492] T a2976e0cb2ab4d7980ec185f2cfba170 P 273b623cdea64c319e0d540e935ba261: Bootstrap starting.
I20260927 18:43:04.789629 31914 log.cc:824] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:04.789695 31915 tablet_bootstrap.cc:654] T a2976e0cb2ab4d7980ec185f2cfba170 P 273b623cdea64c319e0d540e935ba261: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:04.789788 31916 tablet_bootstrap.cc:492] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2: Bootstrap starting.
I20260927 18:43:04.790242 31915 log.cc:824] T a2976e0cb2ab4d7980ec185f2cfba170 P 273b623cdea64c319e0d540e935ba261: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:04.790652 31916 tablet_bootstrap.cc:654] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:04.791005 31914 tablet_bootstrap.cc:492] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162: No bootstrap required, opened a new log
I20260927 18:43:04.791126 31914 ts_tablet_manager.cc:1397] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:43:04.791303 31915 tablet_bootstrap.cc:492] T a2976e0cb2ab4d7980ec185f2cfba170 P 273b623cdea64c319e0d540e935ba261: No bootstrap required, opened a new log
I20260927 18:43:04.791368 31916 log.cc:824] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:04.791393 31915 ts_tablet_manager.cc:1397] T a2976e0cb2ab4d7980ec185f2cfba170 P 273b623cdea64c319e0d540e935ba261: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260927 18:43:04.792268 31916 tablet_bootstrap.cc:492] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2: No bootstrap required, opened a new log
I20260927 18:43:04.792357 31916 ts_tablet_manager.cc:1397] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:43:04.792927 31914 raft_consensus.cc:348] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8cbd20d78554521940ebfcecd87bbd2" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 32921 } } peers { permanent_uuid: "4c3240babd454d09b1bd923aec0ea162" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43003 } } peers { permanent_uuid: "273b623cdea64c319e0d540e935ba261" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45557 } }
I20260927 18:43:04.793097 31914 raft_consensus.cc:374] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:04.793148 31914 raft_consensus.cc:729] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c3240babd454d09b1bd923aec0ea162, State: Initialized, Role: FOLLOWER
I20260927 18:43:04.793289 31914 consensus_queue.cc:260] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8cbd20d78554521940ebfcecd87bbd2" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 32921 } } peers { permanent_uuid: "4c3240babd454d09b1bd923aec0ea162" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43003 } } peers { permanent_uuid: "273b623cdea64c319e0d540e935ba261" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45557 } }
I20260927 18:43:04.793612 31914 ts_tablet_manager.cc:1428] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 18:43:04.793537 31915 raft_consensus.cc:348] T a2976e0cb2ab4d7980ec185f2cfba170 P 273b623cdea64c319e0d540e935ba261 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8cbd20d78554521940ebfcecd87bbd2" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 32921 } } peers { permanent_uuid: "4c3240babd454d09b1bd923aec0ea162" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43003 } } peers { permanent_uuid: "273b623cdea64c319e0d540e935ba261" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45557 } }
I20260927 18:43:04.793754 31629 heartbeater.cc:503] Master 127.11.251.254:35675 was elected leader, sending a full tablet report...
I20260927 18:43:04.793769 31915 raft_consensus.cc:374] T a2976e0cb2ab4d7980ec185f2cfba170 P 273b623cdea64c319e0d540e935ba261 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:04.793835 31915 raft_consensus.cc:729] T a2976e0cb2ab4d7980ec185f2cfba170 P 273b623cdea64c319e0d540e935ba261 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273b623cdea64c319e0d540e935ba261, State: Initialized, Role: FOLLOWER
I20260927 18:43:04.793994 31915 consensus_queue.cc:260] T a2976e0cb2ab4d7980ec185f2cfba170 P 273b623cdea64c319e0d540e935ba261 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8cbd20d78554521940ebfcecd87bbd2" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 32921 } } peers { permanent_uuid: "4c3240babd454d09b1bd923aec0ea162" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43003 } } peers { permanent_uuid: "273b623cdea64c319e0d540e935ba261" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45557 } }
I20260927 18:43:04.794329 31915 ts_tablet_manager.cc:1428] T a2976e0cb2ab4d7980ec185f2cfba170 P 273b623cdea64c319e0d540e935ba261: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260927 18:43:04.794363 31897 heartbeater.cc:503] Master 127.11.251.254:35675 was elected leader, sending a full tablet report...
I20260927 18:43:04.794595 31916 raft_consensus.cc:348] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8cbd20d78554521940ebfcecd87bbd2" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 32921 } } peers { permanent_uuid: "4c3240babd454d09b1bd923aec0ea162" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43003 } } peers { permanent_uuid: "273b623cdea64c319e0d540e935ba261" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45557 } }
I20260927 18:43:04.794775 31916 raft_consensus.cc:374] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:04.794819 31916 raft_consensus.cc:729] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8cbd20d78554521940ebfcecd87bbd2, State: Initialized, Role: FOLLOWER
I20260927 18:43:04.794955 31916 consensus_queue.cc:260] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8cbd20d78554521940ebfcecd87bbd2" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 32921 } } peers { permanent_uuid: "4c3240babd454d09b1bd923aec0ea162" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43003 } } peers { permanent_uuid: "273b623cdea64c319e0d540e935ba261" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45557 } }
I20260927 18:43:04.795295 31916 ts_tablet_manager.cc:1428] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:43:04.795416 31763 heartbeater.cc:503] Master 127.11.251.254:35675 was elected leader, sending a full tablet report...
W20260927 18:43:04.797729 31764 tablet.cc:2369] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:43:04.827064 31920 raft_consensus.cc:482] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:43:04.827190 31920 raft_consensus.cc:504] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8cbd20d78554521940ebfcecd87bbd2" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 32921 } } peers { permanent_uuid: "4c3240babd454d09b1bd923aec0ea162" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43003 } } peers { permanent_uuid: "273b623cdea64c319e0d540e935ba261" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45557 } }
I20260927 18:43:04.827641 31920 leader_election.cc:288] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8cbd20d78554521940ebfcecd87bbd2 (127.11.251.194:32921), 273b623cdea64c319e0d540e935ba261 (127.11.251.195:45557)
I20260927 18:43:04.830945 31911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:04.828038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:53367 (local address 127.11.251.195:45557)
0927 18:43:04.828086 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:04.828088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:04.828309 (+ 221us) server_negotiation.cc:407] Connection header received
0927 18:43:04.828373 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:04.828373 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:04.828383 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:04.828433 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:04.828602 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.829106 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.830198 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.830573 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.830672 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:04.830682 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:04.830833 (+ 151us) server_negotiation.cc:299] Negotiation successful
0927 18:43:04.830872 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 18:43:04.830973 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:04.827970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:45557 (local address 127.11.251.193:53367)
0927 18:43:04.828225 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:04.828235 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:04.828316 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:04.828436 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:04.828436 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:04.828438 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:04.828522 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:04.828525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.829197 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.829199 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:04.830099 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:04.830101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.830677 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.830678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:04.830754 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:04.830758 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:43:04.830827 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 18:43:04.830872 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":2,"thread_start_us":190,"threads_started":1}
I20260927 18:43:04.831065 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:04.827864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:50777 (local address 127.11.251.194:32921)
0927 18:43:04.827954 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:04.827955 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:04.827963 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:43:04.827994 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:04.827995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:04.828003 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:04.828066 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:04.828353 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.828942 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.830143 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.830608 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.830709 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:04.830721 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:04.830889 (+ 168us) server_negotiation.cc:299] Negotiation successful
0927 18:43:04.830944 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260927 18:43:04.831323 31631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:04.827774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:32921 (local address 127.11.251.193:50777)
0927 18:43:04.827832 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:04.827845 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:04.827889 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:04.828122 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:04.828124 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:04.828127 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:04.828241 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:04.828244 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.829052 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.829053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:04.830029 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:04.830031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.830705 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.830707 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:04.830803 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:04.830807 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:43:04.830882 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 18:43:04.830943 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":32}
I20260927 18:43:04.831313 31851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2976e0cb2ab4d7980ec185f2cfba170" candidate_uuid: "4c3240babd454d09b1bd923aec0ea162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273b623cdea64c319e0d540e935ba261" is_pre_election: true
I20260927 18:43:04.831496 31851 raft_consensus.cc:2454] T a2976e0cb2ab4d7980ec185f2cfba170 P 273b623cdea64c319e0d540e935ba261 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c3240babd454d09b1bd923aec0ea162 in term 0.
I20260927 18:43:04.831724 31717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2976e0cb2ab4d7980ec185f2cfba170" candidate_uuid: "4c3240babd454d09b1bd923aec0ea162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8cbd20d78554521940ebfcecd87bbd2" is_pre_election: true
I20260927 18:43:04.831758 31517 leader_election.cc:302] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 273b623cdea64c319e0d540e935ba261, 4c3240babd454d09b1bd923aec0ea162; no voters:
I20260927 18:43:04.831912 31717 raft_consensus.cc:2454] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c3240babd454d09b1bd923aec0ea162 in term 0.
I20260927 18:43:04.831957 31920 raft_consensus.cc:2781] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:43:04.832042 31920 raft_consensus.cc:482] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:43:04.832082 31920 raft_consensus.cc:3037] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:04.833065 31920 raft_consensus.cc:504] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8cbd20d78554521940ebfcecd87bbd2" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 32921 } } peers { permanent_uuid: "4c3240babd454d09b1bd923aec0ea162" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43003 } } peers { permanent_uuid: "273b623cdea64c319e0d540e935ba261" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45557 } }
I20260927 18:43:04.833248 31920 leader_election.cc:288] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162 [CANDIDATE]: Term 1 election: Requested vote from peers d8cbd20d78554521940ebfcecd87bbd2 (127.11.251.194:32921), 273b623cdea64c319e0d540e935ba261 (127.11.251.195:45557)
I20260927 18:43:04.833393 31851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2976e0cb2ab4d7980ec185f2cfba170" candidate_uuid: "4c3240babd454d09b1bd923aec0ea162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273b623cdea64c319e0d540e935ba261"
I20260927 18:43:04.833451 31717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2976e0cb2ab4d7980ec185f2cfba170" candidate_uuid: "4c3240babd454d09b1bd923aec0ea162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8cbd20d78554521940ebfcecd87bbd2"
I20260927 18:43:04.833490 31851 raft_consensus.cc:3037] T a2976e0cb2ab4d7980ec185f2cfba170 P 273b623cdea64c319e0d540e935ba261 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:04.833542 31717 raft_consensus.cc:3037] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:04.834412 31851 raft_consensus.cc:2454] T a2976e0cb2ab4d7980ec185f2cfba170 P 273b623cdea64c319e0d540e935ba261 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c3240babd454d09b1bd923aec0ea162 in term 1.
I20260927 18:43:04.834444 31717 raft_consensus.cc:2454] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c3240babd454d09b1bd923aec0ea162 in term 1.
I20260927 18:43:04.834693 31519 leader_election.cc:302] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c3240babd454d09b1bd923aec0ea162, d8cbd20d78554521940ebfcecd87bbd2; no voters:
I20260927 18:43:04.834837 31920 raft_consensus.cc:2781] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:04.835072 31920 raft_consensus.cc:686] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162 [term 1 LEADER]: Becoming Leader. State: Replica: 4c3240babd454d09b1bd923aec0ea162, State: Running, Role: LEADER
I20260927 18:43:04.835198 31920 consensus_queue.cc:237] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8cbd20d78554521940ebfcecd87bbd2" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 32921 } } peers { permanent_uuid: "4c3240babd454d09b1bd923aec0ea162" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43003 } } peers { permanent_uuid: "273b623cdea64c319e0d540e935ba261" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45557 } }
I20260927 18:43:04.836292 31438 catalog_manager.cc:5495] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c3240babd454d09b1bd923aec0ea162 (127.11.251.193). New cstate: current_term: 1 leader_uuid: "4c3240babd454d09b1bd923aec0ea162" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8cbd20d78554521940ebfcecd87bbd2" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 32921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c3240babd454d09b1bd923aec0ea162" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 43003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "273b623cdea64c319e0d540e935ba261" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45557 } health_report { overall_health: UNKNOWN } } }
I20260927 18:43:04.856691 31912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:04.854182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38448 (local address 127.11.251.193:43003)
0927 18:43:04.854247 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:04.854249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:04.854297 (+ 48us) server_negotiation.cc:407] Connection header received
0927 18:43:04.854328 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:04.854329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:04.854348 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:04.854408 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:04.854632 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.855129 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.855953 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:04.856153 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:04.856225 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:04.856379 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:04.856487 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:04.856554 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:04.856599 (+ 45us) server_negotiation.cc:299] Negotiation successful
0927 18:43:04.856622 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:43:04.859007 31851 raft_consensus.cc:1261] T a2976e0cb2ab4d7980ec185f2cfba170 P 273b623cdea64c319e0d540e935ba261 [term 1 FOLLOWER]: Refusing update from remote peer 4c3240babd454d09b1bd923aec0ea162: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:04.859066 31717 raft_consensus.cc:1261] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2 [term 1 FOLLOWER]: Refusing update from remote peer 4c3240babd454d09b1bd923aec0ea162: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:04.859459 31924 consensus_queue.cc:1035] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273b623cdea64c319e0d540e935ba261" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 45557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:04.859644 31920 consensus_queue.cc:1035] T a2976e0cb2ab4d7980ec185f2cfba170 P 4c3240babd454d09b1bd923aec0ea162 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8cbd20d78554521940ebfcecd87bbd2" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 32921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:04.862144 31927 mvcc.cc:205] Tried to move back new op lower bound from 7334029659579011072 to 7334029659485843456. Current Snapshot: MvccSnapshot[applied={T|T < 7334029659579011072}]
I20260927 18:43:04.864245 31931 mvcc.cc:205] Tried to move back new op lower bound from 7334029659579011072 to 7334029659485843456. Current Snapshot: MvccSnapshot[applied={T|T < 7334029659579011072}]
I20260927 18:43:04.865841 31438 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:43:04.865964 31438 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:43:04.866917 31438 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6729314ff6484aa882582b25fe7ce8ef: Sending DeleteTablet for 3 replicas of tablet a2976e0cb2ab4d7980ec185f2cfba170
I20260927 18:43:04.867312 31831 tablet_service.cc:1514] Processing DeleteTablet for tablet a2976e0cb2ab4d7980ec185f2cfba170 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:04 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:41514
I20260927 18:43:04.867302 31697 tablet_service.cc:1514] Processing DeleteTablet for tablet a2976e0cb2ab4d7980ec185f2cfba170 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:04 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:59390
I20260927 18:43:04.867694 31937 tablet_replica.cc:330] T a2976e0cb2ab4d7980ec185f2cfba170 P 273b623cdea64c319e0d540e935ba261: stopping tablet replica
I20260927 18:43:04.867687 31563 tablet_service.cc:1514] Processing DeleteTablet for tablet a2976e0cb2ab4d7980ec185f2cfba170 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:04 UTC) from {username='kudu', principal='kudu/127.11.251.254@KRBTEST.COM'} at 127.0.0.1:38434
I20260927 18:43:04.867812 31937 raft_consensus.cc:2229] T a2976e0cb2ab4d7980ec185f2cfba170 P 273b623cdea64c319e0d540e935ba261 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:43:04.867941 31937 raft_consensus.cc:2258] T a2976e0cb2ab4d7980ec185f2cfba170 P 273b623cdea64c319e0d540e935ba261 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:04.868109 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 31500
I20260927 18:43:04.868973 31937 ts_tablet_manager.cc:1905] T a2976e0cb2ab4d7980ec185f2cfba170 P 273b623cdea64c319e0d540e935ba261: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:04.871429 31937 ts_tablet_manager.cc:1918] T a2976e0cb2ab4d7980ec185f2cfba170 P 273b623cdea64c319e0d540e935ba261: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:43:04.871511 31937 log.cc:1196] T a2976e0cb2ab4d7980ec185f2cfba170 P 273b623cdea64c319e0d540e935ba261: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/a2976e0cb2ab4d7980ec185f2cfba170
I20260927 18:43:04.871810 31937 ts_tablet_manager.cc:1939] T a2976e0cb2ab4d7980ec185f2cfba170 P 273b623cdea64c319e0d540e935ba261: Deleting consensus metadata
I20260927 18:43:04.872045 31938 tablet_replica.cc:330] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2: stopping tablet replica
I20260927 18:43:04.872174 31938 raft_consensus.cc:2229] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:43:04.872295 31938 raft_consensus.cc:2258] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:04.872874 31423 catalog_manager.cc:4841] TS 273b623cdea64c319e0d540e935ba261 (127.11.251.195:45557): tablet a2976e0cb2ab4d7980ec185f2cfba170 (table test-table [id=b286634779c94045bba024eca57a6add]) successfully deleted
I20260927 18:43:04.872915 31938 ts_tablet_manager.cc:1905] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:04.874481 31938 ts_tablet_manager.cc:1918] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:43:04.874576 31938 log.cc:1196] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/a2976e0cb2ab4d7980ec185f2cfba170
I20260927 18:43:04.874840 31938 ts_tablet_manager.cc:1939] T a2976e0cb2ab4d7980ec185f2cfba170 P d8cbd20d78554521940ebfcecd87bbd2: Deleting consensus metadata
I20260927 18:43:04.875303 31425 catalog_manager.cc:4841] TS d8cbd20d78554521940ebfcecd87bbd2 (127.11.251.194:32921): tablet a2976e0cb2ab4d7980ec185f2cfba170 (table test-table [id=b286634779c94045bba024eca57a6add]) successfully deleted
W20260927 18:43:04.876688 31424 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.251.193:43003 (error 108)
I20260927 18:43:04.877170 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 31634
I20260927 18:43:04.877388 31908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:04.877225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:43003 (local address 127.0.0.1:38456)
0927 18:43:04.877297 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:04.877344 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.193:43003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260927 18:43:04.880929 31424 catalog_manager.cc:4584] TS 4c3240babd454d09b1bd923aec0ea162 (127.11.251.193:43003): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a2976e0cb2ab4d7980ec185f2cfba170: Network error: Client connection negotiation failed: client connection to 127.11.251.193:43003: connect: Connection refused (error 111)
I20260927 18:43:04.883435 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 31768
I20260927 18:43:04.889194 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 31406
2026-09-27T18:43:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2019 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-27T18:43:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:04Z Disabled control of system clock
I20260927 18:43:04.919456 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:45349
--webserver_interface=127.11.251.254
--webserver_port=0
--builtin_ntp_servers=127.11.251.212:37943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:45349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:43:05.008708 31944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:05.008948 31944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:05.008970 31944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:05.010272 31944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:05.010320 31944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:05.010334 31944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:05.010346 31944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:05.010360 31944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:05.011754 31944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:37943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:45349
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:45349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:05.012037 31944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:05.012408 31944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:05.017665 31949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:05.017690 31950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:05.017668 31952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:05.018074 31944 server_base.cc:1034] running on GCE node
I20260927 18:43:05.018338 31944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:05.018700 31944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:05.019877 31944 hybrid_clock.cc:648] HybridClock initialized: now 1790534585019844 us; error 47 us; skew 500 ppm
I20260927 18:43:05.021251 31944 webserver.cc:460] Webserver started at http://127.11.251.254:38269/ using document root <none> and password file <none>
I20260927 18:43:05.021548 31944 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:05.021677 31944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:05.021847 31944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:05.023030 31944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "ffc671ecb1764434993f5a0968112d8a"
format_stamp: "Formatted at 2026-09-27 18:43:05 on dist-test-slave-z47j"
server_key: "ea02037444a44a70a302d9381d012103"
server_key_iv: "901a6fe8059a3c76634f18455f802751"
server_key_version: "encryptionkey@0"
I20260927 18:43:05.023427 31944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "ffc671ecb1764434993f5a0968112d8a"
format_stamp: "Formatted at 2026-09-27 18:43:05 on dist-test-slave-z47j"
server_key: "ea02037444a44a70a302d9381d012103"
server_key_iv: "901a6fe8059a3c76634f18455f802751"
server_key_version: "encryptionkey@0"
I20260927 18:43:05.024904 31944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 18:43:05.026117 31958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:05.026449 31944 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:05.026576 31944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "ffc671ecb1764434993f5a0968112d8a"
format_stamp: "Formatted at 2026-09-27 18:43:05 on dist-test-slave-z47j"
server_key: "ea02037444a44a70a302d9381d012103"
server_key_iv: "901a6fe8059a3c76634f18455f802751"
server_key_version: "encryptionkey@0"
I20260927 18:43:05.026660 31944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:05.057446 31944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:05.057914 31944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:05.058092 31944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:05.063182 31944 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:45349
I20260927 18:43:05.063253 32010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:45349 every 8 connection(s)
I20260927 18:43:05.063663 31944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:05.064850 32011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:05.065238 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 31944
I20260927 18:43:05.065371 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:05.065706 12271 external_mini_cluster.cc:1428] Setting key c028295e6e8e605a8928f312372b0b29
I20260927 18:43:05.068396 32011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a: Bootstrap starting.
I20260927 18:43:05.069123 32011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:05.069440 32011 log.cc:824] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:05.070218 32011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a: No bootstrap required, opened a new log
I20260927 18:43:05.072525 32014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:05.066397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35720 (local address 127.11.251.254:45349)
0927 18:43:05.066673 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:05.066684 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:05.066723 (+ 39us) server_negotiation.cc:407] Connection header received
0927 18:43:05.067265 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:05.067274 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:05.067354 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:05.067532 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:05.067780 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.069148 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.070165 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.070768 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.070896 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:05.071716 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:05.071720 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:05.071720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:05.071963 (+ 243us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:05.072015 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:05.072033 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:05.072133 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:05.072205 (+ 72us) server_negotiation.cc:299] Negotiation successful
0927 18:43:05.072307 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":105,"threads_started":1}
I20260927 18:43:05.072511 32011 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc671ecb1764434993f5a0968112d8a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 45349 } }
I20260927 18:43:05.072695 32011 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:05.072770 32011 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffc671ecb1764434993f5a0968112d8a, State: Initialized, Role: FOLLOWER
I20260927 18:43:05.072906 32011 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc671ecb1764434993f5a0968112d8a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 45349 } }
I20260927 18:43:05.072994 32011 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:05.073028 32011 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:05.073065 32011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:05.073829 32011 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc671ecb1764434993f5a0968112d8a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 45349 } }
I20260927 18:43:05.073926 32011 leader_election.cc:302] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffc671ecb1764434993f5a0968112d8a; no voters:
I20260927 18:43:05.074072 32011 leader_election.cc:288] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:05.074172 32016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:05.074357 32016 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a [term 1 LEADER]: Becoming Leader. State: Replica: ffc671ecb1764434993f5a0968112d8a, State: Running, Role: LEADER
I20260927 18:43:05.074376 32011 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:05.074472 32016 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc671ecb1764434993f5a0968112d8a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 45349 } }
I20260927 18:43:05.074949 32018 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffc671ecb1764434993f5a0968112d8a. Latest consensus state: current_term: 1 leader_uuid: "ffc671ecb1764434993f5a0968112d8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc671ecb1764434993f5a0968112d8a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 45349 } } }
I20260927 18:43:05.074930 32017 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffc671ecb1764434993f5a0968112d8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc671ecb1764434993f5a0968112d8a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 45349 } } }
I20260927 18:43:05.075169 32017 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:05.075454 32018 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:05.075493 32025 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:05.076040 32025 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:05.077781 32025 catalog_manager.cc:1347] Generated new cluster ID: c0dc520c254c442ea611f8adeed08f1d
I20260927 18:43:05.077849 32025 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:05.083285 32025 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:05.083855 32025 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:05.090323 32025 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a: Generated new TSK 0
I20260927 18:43:05.090525 32025 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:05.092831 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.193:0
--local_ip_for_outbound_sockets=127.11.251.193
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:45349
--builtin_ntp_servers=127.11.251.212:37943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:43:05.184053 32035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:05.184288 32035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:05.184311 32035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:05.185521 32035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:05.185566 32035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:05.185655 32035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.193
I20260927 18:43:05.187090 32035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:37943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.251.193
--webserver_port=0
--tserver_master_addrs=127.11.251.254:45349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.193
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:05.187379 32035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:05.187709 32035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:05.193614 32040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:05.193637 32041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:05.193632 32043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:05.194154 32035 server_base.cc:1034] running on GCE node
I20260927 18:43:05.194465 32035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:05.194818 32035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:05.196018 32035 hybrid_clock.cc:648] HybridClock initialized: now 1790534585195991 us; error 50 us; skew 500 ppm
I20260927 18:43:05.197557 32035 webserver.cc:460] Webserver started at http://127.11.251.193:42375/ using document root <none> and password file <none>
I20260927 18:43:05.197896 32035 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:05.197973 32035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:05.198149 32035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:05.199430 32035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-0/data/instance:
uuid: "d546341dffc4476e9fb04473c9eff146"
format_stamp: "Formatted at 2026-09-27 18:43:05 on dist-test-slave-z47j"
server_key: "07e4820f64d6b141fac5056e7e1103c2"
server_key_iv: "b38b962a306ecc44c990ba0f108424d3"
server_key_version: "encryptionkey@0"
I20260927 18:43:05.199836 32035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance:
uuid: "d546341dffc4476e9fb04473c9eff146"
format_stamp: "Formatted at 2026-09-27 18:43:05 on dist-test-slave-z47j"
server_key: "07e4820f64d6b141fac5056e7e1103c2"
server_key_iv: "b38b962a306ecc44c990ba0f108424d3"
server_key_version: "encryptionkey@0"
I20260927 18:43:05.201388 32035 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:43:05.202361 32049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:05.202636 32035 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:43:05.202749 32035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-0/wal
uuid: "d546341dffc4476e9fb04473c9eff146"
format_stamp: "Formatted at 2026-09-27 18:43:05 on dist-test-slave-z47j"
server_key: "07e4820f64d6b141fac5056e7e1103c2"
server_key_iv: "b38b962a306ecc44c990ba0f108424d3"
server_key_version: "encryptionkey@0"
I20260927 18:43:05.202847 32035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:05.234268 32035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:05.234579 32035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:05.234701 32035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:05.234918 32035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:05.235312 32035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:05.235352 32035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:05.235416 32035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:05.235448 32035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:05.242167 32035 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.193:37843
I20260927 18:43:05.242282 32162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.193:37843 every 8 connection(s)
I20260927 18:43:05.242620 32035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-0/data/info.pb
I20260927 18:43:05.248771 32014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:05.243553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.193:56211 (local address 127.11.251.254:45349)
0927 18:43:05.243650 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:05.243653 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:05.243921 (+ 268us) server_negotiation.cc:407] Connection header received
0927 18:43:05.244571 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:05.244572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:05.244592 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:05.244652 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:05.245447 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.246091 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.247158 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.247372 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.247446 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:05.248445 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:05.248447 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:05.248447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:05.248453 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:05.248475 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:05.248479 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:05.248542 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:05.248639 (+ 97us) server_negotiation.cc:299] Negotiation successful
0927 18:43:05.248673 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:43:05.249038 32165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:05.243515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:45349 (local address 127.11.251.193:56211)
0927 18:43:05.243763 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:05.243806 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:05.244454 (+ 648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:05.244730 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:05.244735 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:05.244753 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:05.245325 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:05.245329 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.246189 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.246190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:05.247054 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:05.247056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.247466 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.247467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:05.247613 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:05.248241 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:43:05.248242 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:05.248274 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:43:05.248298 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:05.248542 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:05.248549 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:05.248552 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:43:05.248643 (+ 91us) client_negotiation.cc:240] Negotiation successful
0927 18:43:05.248751 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":96,"threads_started":1}
I20260927 18:43:05.249624 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 32035
I20260927 18:43:05.249771 32163 heartbeater.cc:347] Connected to a master server at 127.11.251.254:45349
I20260927 18:43:05.249807 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-0/wal/instance
I20260927 18:43:05.249881 32163 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:05.250034 12271 external_mini_cluster.cc:1428] Setting key 2dcea8254efc9b6bd0ef2f44543b29e8
I20260927 18:43:05.250114 32163 heartbeater.cc:511] Master 127.11.251.254:45349 requested a full tablet report, sending...
I20260927 18:43:05.250850 31974 ts_manager.cc:194] Registered new tserver with Master: d546341dffc4476e9fb04473c9eff146 (127.11.251.193:37843)
I20260927 18:43:05.251187 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.194:0
--local_ip_for_outbound_sockets=127.11.251.194
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:45349
--builtin_ntp_servers=127.11.251.212:37943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 18:43:05.251827 31974 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.251.193:56211
W20260927 18:43:05.340802 32166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:05.341050 32166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:05.341073 32166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:05.342348 32166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:05.342397 32166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:05.342423 32166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.194
I20260927 18:43:05.343824 32166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:37943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.251.194
--webserver_port=0
--tserver_master_addrs=127.11.251.254:45349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.194
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:05.344101 32166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:05.344453 32166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:05.349960 32172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:05.349941 32171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:05.349941 32174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:05.350497 32166 server_base.cc:1034] running on GCE node
I20260927 18:43:05.350713 32166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:05.350937 32166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:05.352085 32166 hybrid_clock.cc:648] HybridClock initialized: now 1790534585352072 us; error 40 us; skew 500 ppm
I20260927 18:43:05.353415 32166 webserver.cc:460] Webserver started at http://127.11.251.194:38921/ using document root <none> and password file <none>
I20260927 18:43:05.353736 32166 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:05.353809 32166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:05.353984 32166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:05.355201 32166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-1/data/instance:
uuid: "3b5592fb7aa74d1fa00a07b4f03e45b4"
format_stamp: "Formatted at 2026-09-27 18:43:05 on dist-test-slave-z47j"
server_key: "c4ec0f8fecc29770a781745bb5da661c"
server_key_iv: "570023a4aa941ed6cd075ae54ea987ed"
server_key_version: "encryptionkey@0"
I20260927 18:43:05.355602 32166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance:
uuid: "3b5592fb7aa74d1fa00a07b4f03e45b4"
format_stamp: "Formatted at 2026-09-27 18:43:05 on dist-test-slave-z47j"
server_key: "c4ec0f8fecc29770a781745bb5da661c"
server_key_iv: "570023a4aa941ed6cd075ae54ea987ed"
server_key_version: "encryptionkey@0"
I20260927 18:43:05.357088 32166 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:43:05.358114 32180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:05.358429 32166 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 18:43:05.358559 32166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-1/wal
uuid: "3b5592fb7aa74d1fa00a07b4f03e45b4"
format_stamp: "Formatted at 2026-09-27 18:43:05 on dist-test-slave-z47j"
server_key: "c4ec0f8fecc29770a781745bb5da661c"
server_key_iv: "570023a4aa941ed6cd075ae54ea987ed"
server_key_version: "encryptionkey@0"
I20260927 18:43:05.358666 32166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:05.383773 32166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:05.384285 32166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:05.384424 32166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:05.384709 32166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:05.385195 32166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:05.385236 32166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:05.385263 32166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:05.385275 32166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:05.393186 32166 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.194:35565
I20260927 18:43:05.393306 32293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.194:35565 every 8 connection(s)
I20260927 18:43:05.393744 32166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-1/data/info.pb
I20260927 18:43:05.397297 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 32166
I20260927 18:43:05.397425 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-1/wal/instance
I20260927 18:43:05.397699 12271 external_mini_cluster.cc:1428] Setting key eec625a5c6e8bd5a8dab5e719ff04c36
I20260927 18:43:05.399121 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.251.195:0
--local_ip_for_outbound_sockets=127.11.251.195
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:45349
--builtin_ntp_servers=127.11.251.212:37943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 18:43:05.401013 32014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:05.394701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.194:41569 (local address 127.11.251.254:45349)
0927 18:43:05.394757 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:05.394760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:05.395046 (+ 286us) server_negotiation.cc:407] Connection header received
0927 18:43:05.395730 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:05.395731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:05.395743 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:05.395787 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:05.396716 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.397365 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.398944 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.399280 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.399362 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:05.400590 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:05.400593 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:05.400594 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:05.400602 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:05.400632 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:05.400639 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:05.400719 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:05.400839 (+ 120us) server_negotiation.cc:299] Negotiation successful
0927 18:43:05.400891 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 18:43:05.401149 32296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:05.394658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:45349 (local address 127.11.251.194:41569)
0927 18:43:05.394891 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:05.394931 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:05.395557 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:05.395878 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:05.395882 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:05.395891 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:05.396590 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:05.396597 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.397512 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.397514 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:05.398828 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:05.398833 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.399447 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.399448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:05.399677 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:05.400360 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:43:05.400361 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:05.400403 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:43:05.400435 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:05.400730 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:05.400736 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:05.400739 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:43:05.400816 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 18:43:05.400937 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
I20260927 18:43:05.401774 32294 heartbeater.cc:347] Connected to a master server at 127.11.251.254:45349
I20260927 18:43:05.401886 32294 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:05.402094 32294 heartbeater.cc:511] Master 127.11.251.254:45349 requested a full tablet report, sending...
I20260927 18:43:05.402590 31974 ts_manager.cc:194] Registered new tserver with Master: 3b5592fb7aa74d1fa00a07b4f03e45b4 (127.11.251.194:35565)
I20260927 18:43:05.403108 31974 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.251.194:41569
W20260927 18:43:05.492641 32297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:05.492889 32297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:05.492924 32297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:05.494222 32297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:05.494292 32297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:05.494335 32297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.251.195
I20260927 18:43:05.495697 32297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:37943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.251.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.251.195
--webserver_port=0
--tserver_master_addrs=127.11.251.254:45349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.251.195
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:05.495966 32297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:05.496263 32297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:05.501427 32303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:05.501520 32302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:05.501502 32305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:05.501914 32297 server_base.cc:1034] running on GCE node
I20260927 18:43:05.502162 32297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:05.502439 32297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:05.503625 32297 hybrid_clock.cc:648] HybridClock initialized: now 1790534585503611 us; error 29 us; skew 500 ppm
I20260927 18:43:05.504904 32297 webserver.cc:460] Webserver started at http://127.11.251.195:36665/ using document root <none> and password file <none>
I20260927 18:43:05.505249 32297 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:05.505304 32297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:05.505396 32297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:05.506521 32297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-2/data/instance:
uuid: "35654bb74e194fcdb0b6f8185e47daba"
format_stamp: "Formatted at 2026-09-27 18:43:05 on dist-test-slave-z47j"
server_key: "cc23fbc6b41ccb99c218bbe8749e9ed9"
server_key_iv: "40d9bd56aa429cbaeb48ab9f9c37c681"
server_key_version: "encryptionkey@0"
I20260927 18:43:05.506819 32297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance:
uuid: "35654bb74e194fcdb0b6f8185e47daba"
format_stamp: "Formatted at 2026-09-27 18:43:05 on dist-test-slave-z47j"
server_key: "cc23fbc6b41ccb99c218bbe8749e9ed9"
server_key_iv: "40d9bd56aa429cbaeb48ab9f9c37c681"
server_key_version: "encryptionkey@0"
I20260927 18:43:05.508147 32297 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 18:43:05.508894 32311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:05.509153 32297 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:05.509262 32297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-2/wal
uuid: "35654bb74e194fcdb0b6f8185e47daba"
format_stamp: "Formatted at 2026-09-27 18:43:05 on dist-test-slave-z47j"
server_key: "cc23fbc6b41ccb99c218bbe8749e9ed9"
server_key_iv: "40d9bd56aa429cbaeb48ab9f9c37c681"
server_key_version: "encryptionkey@0"
I20260927 18:43:05.509348 32297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:05.526530 32297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:05.526847 32297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:05.527005 32297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:05.527243 32297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:05.527597 32297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:05.527660 32297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:05.527699 32297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:05.527725 32297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:05.535121 32297 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.195:46585
I20260927 18:43:05.535166 32424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.195:46585 every 8 connection(s)
I20260927 18:43:05.535509 32297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-2/data/info.pb
I20260927 18:43:05.541491 32014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:05.536357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:58849 (local address 127.11.251.254:45349)
0927 18:43:05.536466 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:05.536470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:05.536715 (+ 245us) server_negotiation.cc:407] Connection header received
0927 18:43:05.537379 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:05.537380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:05.537394 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:05.537447 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:05.538283 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.538836 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.539897 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.540099 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.540173 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:05.541171 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:05.541173 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:05.541173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:05.541178 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:05.541197 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:05.541205 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:05.541268 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:05.541365 (+ 97us) server_negotiation.cc:299] Negotiation successful
0927 18:43:05.541394 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 18:43:05.541785 32427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:05.536323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:45349 (local address 127.11.251.195:58849)
0927 18:43:05.536565 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:05.536607 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:05.537250 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:05.537543 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:05.537547 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:05.537556 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:05.538177 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:05.538182 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.538940 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.538941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:05.539785 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:05.539787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.540193 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.540194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:05.540352 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:05.541008 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:43:05.541008 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:05.541041 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:43:05.541066 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:05.541267 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:05.541274 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:05.541278 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:43:05.541374 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 18:43:05.541483 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":85,"threads_started":1}
I20260927 18:43:05.542523 32425 heartbeater.cc:347] Connected to a master server at 127.11.251.254:45349
I20260927 18:43:05.542634 32425 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:05.542855 32425 heartbeater.cc:511] Master 127.11.251.254:45349 requested a full tablet report, sending...
I20260927 18:43:05.543284 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 32297
I20260927 18:43:05.543417 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-2/wal/instance
I20260927 18:43:05.543452 31974 ts_manager.cc:194] Registered new tserver with Master: 35654bb74e194fcdb0b6f8185e47daba (127.11.251.195:46585)
I20260927 18:43:05.543664 12271 external_mini_cluster.cc:1428] Setting key e609d1ec9e36e1b3e83291c25eb4b4f3
I20260927 18:43:05.543974 31974 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.251.195:58849
I20260927 18:43:05.544875 12271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:43:05.549719 32014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:05.546533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35726 (local address 127.11.251.254:45349)
0927 18:43:05.546587 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:05.546589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:05.546788 (+ 199us) server_negotiation.cc:407] Connection header received
0927 18:43:05.546870 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:05.546871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:05.546889 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:05.546953 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:05.547129 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.547638 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.548446 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.548636 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.548720 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:05.549418 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:05.549419 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:05.549420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:05.549424 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:05.549444 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:05.549448 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:05.549508 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:05.549570 (+ 62us) server_negotiation.cc:299] Negotiation successful
0927 18:43:05.549625 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 18:43:05.551335 31975 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:43:05.552871 31975 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:43:05.561146 32436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:05.555738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43916 (local address 127.11.251.195:46585)
0927 18:43:05.556001 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:05.556008 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:05.556188 (+ 180us) server_negotiation.cc:407] Connection header received
0927 18:43:05.556356 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:05.556359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:05.556377 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:05.556429 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:05.556754 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.558053 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.559828 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.560652 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.560735 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:05.560829 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:05.561007 (+ 178us) server_negotiation.cc:299] Negotiation successful
0927 18:43:05.561048 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":141,"threads_started":1}
I20260927 18:43:05.561146 32437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:05.555542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.195:46585 (local address 127.0.0.1:43916)
0927 18:43:05.556067 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:05.556123 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:05.556233 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:05.556561 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:05.556564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:05.556567 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:05.556642 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:05.556645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.558155 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.558157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:05.559723 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:05.559726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.560772 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.560773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:05.560858 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:05.560887 (+ 29us) client_negotiation.cc:769] Sending connection context
0927 18:43:05.560974 (+ 87us) client_negotiation.cc:240] Negotiation successful
0927 18:43:05.561045 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":2,"thread_start_us":138,"threads_started":1}
I20260927 18:43:05.561653 32434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:05.555797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:35565 (local address 127.0.0.1:59878)
0927 18:43:05.556444 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:05.556457 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:05.556535 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:05.557737 (+ 1202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:05.557738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:05.557740 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:05.557782 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:05.557784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.559061 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.559062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:05.560161 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:05.560164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.561406 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.561408 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:05.561470 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:05.561474 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:43:05.561520 (+ 46us) client_negotiation.cc:240] Negotiation successful
0927 18:43:05.561567 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":121,"threads_started":1}
I20260927 18:43:05.562012 32438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:05.556061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59878 (local address 127.11.251.194:35565)
0927 18:43:05.556276 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:05.556280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:05.557500 (+ 1220us) server_negotiation.cc:407] Connection header received
0927 18:43:05.557552 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:05.557555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:05.557590 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:05.557671 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:05.557861 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.558960 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.560270 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.561027 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.561096 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:05.561189 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:05.561814 (+ 625us) server_negotiation.cc:299] Negotiation successful
0927 18:43:05.561868 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":111,"threads_started":1}
I20260927 18:43:05.562271 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:05.555698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51810 (local address 127.11.251.193:37843)
0927 18:43:05.556356 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:05.556359 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:05.556674 (+ 315us) server_negotiation.cc:407] Connection header received
0927 18:43:05.556747 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:05.556750 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:05.556808 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:05.556903 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:05.557060 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.558512 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.560873 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.561717 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.561816 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:05.561899 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:05.562131 (+ 232us) server_negotiation.cc:299] Negotiation successful
0927 18:43:05.562187 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":92,"threads_started":1}
I20260927 18:43:05.562301 32359 tablet_service.cc:1467] Processing CreateTablet for tablet 0a1f7ec15941452aabab89729402a29a (DEFAULT_TABLE table=test-table [id=8bca3d84b97d4bc9b66a748ba1e92c9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:05.562261 32433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:05.555538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:37843 (local address 127.0.0.1:51810)
0927 18:43:05.555986 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:05.556597 (+ 611us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:05.556660 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:05.556928 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:05.556929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:05.556932 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:05.556983 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:05.556984 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.558606 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.558607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:05.559727 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:05.559729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.561942 (+ 2213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.561944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:05.562021 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:05.562025 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:43:05.562107 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 18:43:05.562170 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":30,"thread_start_us":85,"threads_started":1}
I20260927 18:43:05.562777 32359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a1f7ec15941452aabab89729402a29a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:05.563277 32228 tablet_service.cc:1467] Processing CreateTablet for tablet 0a1f7ec15941452aabab89729402a29a (DEFAULT_TABLE table=test-table [id=8bca3d84b97d4bc9b66a748ba1e92c9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:05.563632 32228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a1f7ec15941452aabab89729402a29a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:05.566365 32097 tablet_service.cc:1467] Processing CreateTablet for tablet 0a1f7ec15941452aabab89729402a29a (DEFAULT_TABLE table=test-table [id=8bca3d84b97d4bc9b66a748ba1e92c9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:05.566788 32097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a1f7ec15941452aabab89729402a29a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:05.567291 32440 tablet_bootstrap.cc:492] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4: Bootstrap starting.
I20260927 18:43:05.567339 32439 tablet_bootstrap.cc:492] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba: Bootstrap starting.
I20260927 18:43:05.568091 32439 tablet_bootstrap.cc:654] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:05.568091 32440 tablet_bootstrap.cc:654] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:05.568475 32440 log.cc:824] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:05.568475 32439 log.cc:824] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:05.569403 32440 tablet_bootstrap.cc:492] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4: No bootstrap required, opened a new log
I20260927 18:43:05.569519 32440 ts_tablet_manager.cc:1397] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:43:05.569938 32439 tablet_bootstrap.cc:492] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba: No bootstrap required, opened a new log
I20260927 18:43:05.570035 32439 ts_tablet_manager.cc:1397] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 18:43:05.570406 32443 tablet_bootstrap.cc:492] T 0a1f7ec15941452aabab89729402a29a P d546341dffc4476e9fb04473c9eff146: Bootstrap starting.
I20260927 18:43:05.571295 32443 tablet_bootstrap.cc:654] T 0a1f7ec15941452aabab89729402a29a P d546341dffc4476e9fb04473c9eff146: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:05.571714 32443 log.cc:824] T 0a1f7ec15941452aabab89729402a29a P d546341dffc4476e9fb04473c9eff146: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:05.572062 32440 raft_consensus.cc:348] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d546341dffc4476e9fb04473c9eff146" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 37843 } } peers { permanent_uuid: "35654bb74e194fcdb0b6f8185e47daba" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 46585 } } peers { permanent_uuid: "3b5592fb7aa74d1fa00a07b4f03e45b4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 35565 } }
I20260927 18:43:05.572264 32440 raft_consensus.cc:374] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:05.572310 32440 raft_consensus.cc:729] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b5592fb7aa74d1fa00a07b4f03e45b4, State: Initialized, Role: FOLLOWER
I20260927 18:43:05.572444 32440 consensus_queue.cc:260] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d546341dffc4476e9fb04473c9eff146" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 37843 } } peers { permanent_uuid: "35654bb74e194fcdb0b6f8185e47daba" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 46585 } } peers { permanent_uuid: "3b5592fb7aa74d1fa00a07b4f03e45b4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 35565 } }
I20260927 18:43:05.572774 32440 ts_tablet_manager.cc:1428] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260927 18:43:05.572785 32294 heartbeater.cc:503] Master 127.11.251.254:45349 was elected leader, sending a full tablet report...
I20260927 18:43:05.573014 32443 tablet_bootstrap.cc:492] T 0a1f7ec15941452aabab89729402a29a P d546341dffc4476e9fb04473c9eff146: No bootstrap required, opened a new log
I20260927 18:43:05.572976 32439 raft_consensus.cc:348] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d546341dffc4476e9fb04473c9eff146" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 37843 } } peers { permanent_uuid: "35654bb74e194fcdb0b6f8185e47daba" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 46585 } } peers { permanent_uuid: "3b5592fb7aa74d1fa00a07b4f03e45b4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 35565 } }
I20260927 18:43:05.573122 32443 ts_tablet_manager.cc:1397] T 0a1f7ec15941452aabab89729402a29a P d546341dffc4476e9fb04473c9eff146: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:43:05.573153 32439 raft_consensus.cc:374] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:05.573197 32439 raft_consensus.cc:729] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35654bb74e194fcdb0b6f8185e47daba, State: Initialized, Role: FOLLOWER
I20260927 18:43:05.573355 32439 consensus_queue.cc:260] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d546341dffc4476e9fb04473c9eff146" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 37843 } } peers { permanent_uuid: "35654bb74e194fcdb0b6f8185e47daba" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 46585 } } peers { permanent_uuid: "3b5592fb7aa74d1fa00a07b4f03e45b4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 35565 } }
I20260927 18:43:05.573655 32439 ts_tablet_manager.cc:1428] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260927 18:43:05.573891 32425 heartbeater.cc:503] Master 127.11.251.254:45349 was elected leader, sending a full tablet report...
I20260927 18:43:05.575872 32443 raft_consensus.cc:348] T 0a1f7ec15941452aabab89729402a29a P d546341dffc4476e9fb04473c9eff146 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d546341dffc4476e9fb04473c9eff146" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 37843 } } peers { permanent_uuid: "35654bb74e194fcdb0b6f8185e47daba" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 46585 } } peers { permanent_uuid: "3b5592fb7aa74d1fa00a07b4f03e45b4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 35565 } }
I20260927 18:43:05.576049 32443 raft_consensus.cc:374] T 0a1f7ec15941452aabab89729402a29a P d546341dffc4476e9fb04473c9eff146 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:05.576095 32443 raft_consensus.cc:729] T 0a1f7ec15941452aabab89729402a29a P d546341dffc4476e9fb04473c9eff146 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d546341dffc4476e9fb04473c9eff146, State: Initialized, Role: FOLLOWER
I20260927 18:43:05.576225 32443 consensus_queue.cc:260] T 0a1f7ec15941452aabab89729402a29a P d546341dffc4476e9fb04473c9eff146 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d546341dffc4476e9fb04473c9eff146" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 37843 } } peers { permanent_uuid: "35654bb74e194fcdb0b6f8185e47daba" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 46585 } } peers { permanent_uuid: "3b5592fb7aa74d1fa00a07b4f03e45b4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 35565 } }
I20260927 18:43:05.576526 32443 ts_tablet_manager.cc:1428] T 0a1f7ec15941452aabab89729402a29a P d546341dffc4476e9fb04473c9eff146: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:43:05.576597 32163 heartbeater.cc:503] Master 127.11.251.254:45349 was elected leader, sending a full tablet report...
W20260927 18:43:05.644520 32295 tablet.cc:2369] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:43:05.743496 32164 tablet.cc:2369] T 0a1f7ec15941452aabab89729402a29a P d546341dffc4476e9fb04473c9eff146: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:43:05.786139 32426 tablet.cc:2369] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:43:05.958666 32446 raft_consensus.cc:482] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:43:05.958860 32446 raft_consensus.cc:504] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d546341dffc4476e9fb04473c9eff146" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 37843 } } peers { permanent_uuid: "35654bb74e194fcdb0b6f8185e47daba" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 46585 } } peers { permanent_uuid: "3b5592fb7aa74d1fa00a07b4f03e45b4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 35565 } }
I20260927 18:43:05.959277 32446 leader_election.cc:288] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d546341dffc4476e9fb04473c9eff146 (127.11.251.193:37843), 3b5592fb7aa74d1fa00a07b4f03e45b4 (127.11.251.194:35565)
I20260927 18:43:05.963667 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:05.959756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:45055 (local address 127.11.251.193:37843)
0927 18:43:05.959814 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:05.959816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:05.959969 (+ 153us) server_negotiation.cc:407] Connection header received
0927 18:43:05.959998 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:05.959998 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:05.960009 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:05.960072 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:05.960268 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.961022 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.962703 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.963283 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.963356 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:05.963364 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:05.963555 (+ 191us) server_negotiation.cc:299] Negotiation successful
0927 18:43:05.963594 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260927 18:43:05.963667 32438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:05.959762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.251.195:59557 (local address 127.11.251.194:35565)
0927 18:43:05.959814 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:05.959816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:05.959829 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:43:05.959869 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:05.959870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:05.959880 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:05.959999 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:05.960268 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.961022 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.962717 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.963270 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.963356 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:05.963364 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:05.963555 (+ 191us) server_negotiation.cc:299] Negotiation successful
0927 18:43:05.963594 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 18:43:05.963682 32427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:05.959610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.194:35565 (local address 127.11.251.195:59557)
0927 18:43:05.959689 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:05.959705 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:05.959761 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:05.960024 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:05.960025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:05.960027 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:05.960153 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:05.960156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.961128 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.961129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:05.962586 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:05.962589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.963364 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.963366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:05.963447 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:05.963453 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:43:05.963523 (+ 70us) client_negotiation.cc:240] Negotiation successful
0927 18:43:05.963570 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":49}
I20260927 18:43:05.963691 32448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:05.959612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:37843 (local address 127.11.251.195:45055)
0927 18:43:05.959853 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:05.959862 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:05.959922 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:05.960087 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:05.960087 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:05.960089 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:05.960163 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:05.960164 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.961128 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.961129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:05.962621 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:05.962622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:05.963399 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:05.963400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:05.963466 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:05.963469 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:43:05.963527 (+ 58us) client_negotiation.cc:240] Negotiation successful
0927 18:43:05.963567 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":136,"thread_start_us":127,"threads_started":1}
I20260927 18:43:05.964206 32248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1f7ec15941452aabab89729402a29a" candidate_uuid: "35654bb74e194fcdb0b6f8185e47daba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b5592fb7aa74d1fa00a07b4f03e45b4" is_pre_election: true
I20260927 18:43:05.964250 32117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1f7ec15941452aabab89729402a29a" candidate_uuid: "35654bb74e194fcdb0b6f8185e47daba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d546341dffc4476e9fb04473c9eff146" is_pre_election: true
I20260927 18:43:05.964422 32248 raft_consensus.cc:2454] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35654bb74e194fcdb0b6f8185e47daba in term 0.
I20260927 18:43:05.964422 32117 raft_consensus.cc:2454] T 0a1f7ec15941452aabab89729402a29a P d546341dffc4476e9fb04473c9eff146 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35654bb74e194fcdb0b6f8185e47daba in term 0.
I20260927 18:43:05.964706 32314 leader_election.cc:302] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35654bb74e194fcdb0b6f8185e47daba, 3b5592fb7aa74d1fa00a07b4f03e45b4; no voters:
I20260927 18:43:05.964860 32446 raft_consensus.cc:2781] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:43:05.964922 32446 raft_consensus.cc:482] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:43:05.964947 32446 raft_consensus.cc:3037] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:05.965904 32446 raft_consensus.cc:504] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d546341dffc4476e9fb04473c9eff146" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 37843 } } peers { permanent_uuid: "35654bb74e194fcdb0b6f8185e47daba" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 46585 } } peers { permanent_uuid: "3b5592fb7aa74d1fa00a07b4f03e45b4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 35565 } }
I20260927 18:43:05.966073 32446 leader_election.cc:288] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba [CANDIDATE]: Term 1 election: Requested vote from peers d546341dffc4476e9fb04473c9eff146 (127.11.251.193:37843), 3b5592fb7aa74d1fa00a07b4f03e45b4 (127.11.251.194:35565)
I20260927 18:43:05.966295 32248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1f7ec15941452aabab89729402a29a" candidate_uuid: "35654bb74e194fcdb0b6f8185e47daba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b5592fb7aa74d1fa00a07b4f03e45b4"
I20260927 18:43:05.966302 32117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1f7ec15941452aabab89729402a29a" candidate_uuid: "35654bb74e194fcdb0b6f8185e47daba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d546341dffc4476e9fb04473c9eff146"
I20260927 18:43:05.966405 32248 raft_consensus.cc:3037] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:05.966405 32117 raft_consensus.cc:3037] T 0a1f7ec15941452aabab89729402a29a P d546341dffc4476e9fb04473c9eff146 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:05.967434 32117 raft_consensus.cc:2454] T 0a1f7ec15941452aabab89729402a29a P d546341dffc4476e9fb04473c9eff146 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35654bb74e194fcdb0b6f8185e47daba in term 1.
I20260927 18:43:05.967434 32248 raft_consensus.cc:2454] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35654bb74e194fcdb0b6f8185e47daba in term 1.
I20260927 18:43:05.967628 32314 leader_election.cc:302] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35654bb74e194fcdb0b6f8185e47daba, 3b5592fb7aa74d1fa00a07b4f03e45b4; no voters:
I20260927 18:43:05.967759 32446 raft_consensus.cc:2781] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:05.967943 32446 raft_consensus.cc:686] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba [term 1 LEADER]: Becoming Leader. State: Replica: 35654bb74e194fcdb0b6f8185e47daba, State: Running, Role: LEADER
I20260927 18:43:05.968110 32446 consensus_queue.cc:237] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d546341dffc4476e9fb04473c9eff146" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 37843 } } peers { permanent_uuid: "35654bb74e194fcdb0b6f8185e47daba" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 46585 } } peers { permanent_uuid: "3b5592fb7aa74d1fa00a07b4f03e45b4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 35565 } }
I20260927 18:43:05.969146 31975 catalog_manager.cc:5495] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba reported cstate change: term changed from 0 to 1, leader changed from <none> to 35654bb74e194fcdb0b6f8185e47daba (127.11.251.195). New cstate: current_term: 1 leader_uuid: "35654bb74e194fcdb0b6f8185e47daba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d546341dffc4476e9fb04473c9eff146" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 37843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35654bb74e194fcdb0b6f8185e47daba" member_type: VOTER last_known_addr { host: "127.11.251.195" port: 46585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b5592fb7aa74d1fa00a07b4f03e45b4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 35565 } health_report { overall_health: UNKNOWN } } }
I20260927 18:43:06.003629 32436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:06.000532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43920 (local address 127.11.251.195:46585)
0927 18:43:06.000605 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:06.000608 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:06.000618 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:43:06.000678 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:06.000679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:06.000700 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:06.000777 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:06.000930 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:06.001541 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:06.002663 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:06.002885 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:06.002988 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:06.003175 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:06.003300 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:06.003397 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:06.003486 (+ 89us) server_negotiation.cc:299] Negotiation successful
0927 18:43:06.003510 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:43:06.006132 32117 raft_consensus.cc:1261] T 0a1f7ec15941452aabab89729402a29a P d546341dffc4476e9fb04473c9eff146 [term 1 FOLLOWER]: Refusing update from remote peer 35654bb74e194fcdb0b6f8185e47daba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:06.006137 32248 raft_consensus.cc:1261] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4 [term 1 FOLLOWER]: Refusing update from remote peer 35654bb74e194fcdb0b6f8185e47daba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:06.006574 32449 consensus_queue.cc:1035] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba [LEADER]: Connected to new peer: Peer: permanent_uuid: "d546341dffc4476e9fb04473c9eff146" member_type: VOTER last_known_addr { host: "127.11.251.193" port: 37843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:06.006762 32446 consensus_queue.cc:1035] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b5592fb7aa74d1fa00a07b4f03e45b4" member_type: VOTER last_known_addr { host: "127.11.251.194" port: 35565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:06.009472 32452 mvcc.cc:205] Tried to move back new op lower bound from 7334029664277487616 to 7334029664126177280. Current Snapshot: MvccSnapshot[applied={T|T < 7334029664277487616}]
I20260927 18:43:06.009730 32453 mvcc.cc:205] Tried to move back new op lower bound from 7334029664277487616 to 7334029664126177280. Current Snapshot: MvccSnapshot[applied={T|T < 7334029664277487616}]
I20260927 18:43:06.012941 31974 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:43:06.013056 31974 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:43:06.013998 31974 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ffc671ecb1764434993f5a0968112d8a: Sending DeleteTablet for 3 replicas of tablet 0a1f7ec15941452aabab89729402a29a
I20260927 18:43:06.014437 32097 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a1f7ec15941452aabab89729402a29a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:06 UTC) from {username='slave'} at 127.0.0.1:51810
I20260927 18:43:06.014487 32359 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a1f7ec15941452aabab89729402a29a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:06 UTC) from {username='slave'} at 127.0.0.1:43916
I20260927 18:43:06.014504 32228 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a1f7ec15941452aabab89729402a29a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:06 UTC) from {username='slave'} at 127.0.0.1:59878
I20260927 18:43:06.014798 32464 tablet_replica.cc:330] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4: stopping tablet replica
I20260927 18:43:06.014912 32462 tablet_replica.cc:330] T 0a1f7ec15941452aabab89729402a29a P d546341dffc4476e9fb04473c9eff146: stopping tablet replica
I20260927 18:43:06.015231 32463 tablet_replica.cc:330] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba: stopping tablet replica
I20260927 18:43:06.015368 32462 raft_consensus.cc:2229] T 0a1f7ec15941452aabab89729402a29a P d546341dffc4476e9fb04473c9eff146 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:43:06.015443 32463 raft_consensus.cc:2229] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:43:06.015846 32462 raft_consensus.cc:2258] T 0a1f7ec15941452aabab89729402a29a P d546341dffc4476e9fb04473c9eff146 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:06.015929 32463 raft_consensus.cc:2258] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:06.016495 32462 ts_tablet_manager.cc:1905] T 0a1f7ec15941452aabab89729402a29a P d546341dffc4476e9fb04473c9eff146: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:06.016563 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 32035
I20260927 18:43:06.016692 32463 ts_tablet_manager.cc:1905] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:06.018885 32463 ts_tablet_manager.cc:1918] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:43:06.019016 32464 raft_consensus.cc:2229] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:43:06.019198 32464 raft_consensus.cc:2258] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:06.019330 32463 log.cc:1196] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-2/wal/wals/0a1f7ec15941452aabab89729402a29a
I20260927 18:43:06.019716 32463 ts_tablet_manager.cc:1939] T 0a1f7ec15941452aabab89729402a29a P 35654bb74e194fcdb0b6f8185e47daba: Deleting consensus metadata
I20260927 18:43:06.019723 32464 ts_tablet_manager.cc:1905] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:06.021996 32464 ts_tablet_manager.cc:1918] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:43:06.022080 32464 log.cc:1196] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4: Deleting WAL directory at /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534329098323-12271-0/minicluster-data/ts-1/wal/wals/0a1f7ec15941452aabab89729402a29a
I20260927 18:43:06.022329 32464 ts_tablet_manager.cc:1939] T 0a1f7ec15941452aabab89729402a29a P 3b5592fb7aa74d1fa00a07b4f03e45b4: Deleting consensus metadata
I20260927 18:43:06.022980 31961 catalog_manager.cc:4841] TS 3b5592fb7aa74d1fa00a07b4f03e45b4 (127.11.251.194:35565): tablet 0a1f7ec15941452aabab89729402a29a (table test-table [id=8bca3d84b97d4bc9b66a748ba1e92c9b]) successfully deleted
I20260927 18:43:06.023082 31961 catalog_manager.cc:4841] TS 35654bb74e194fcdb0b6f8185e47daba (127.11.251.195:46585): tablet 0a1f7ec15941452aabab89729402a29a (table test-table [id=8bca3d84b97d4bc9b66a748ba1e92c9b]) successfully deleted
W20260927 18:43:06.024905 31959 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.251.193:37843 (error 108)
I20260927 18:43:06.025225 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 32166
I20260927 18:43:06.025470 32437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:06.025290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.193:37843 (local address 127.0.0.1:51818)
0927 18:43:06.025349 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:06.025408 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.193:37843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260927 18:43:06.026059 31959 catalog_manager.cc:4584] TS d546341dffc4476e9fb04473c9eff146 (127.11.251.193:37843): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a1f7ec15941452aabab89729402a29a: Network error: Client connection negotiation failed: client connection to 127.11.251.193:37843: connect: Connection refused (error 111)
I20260927 18:43:06.032476 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 32297
I20260927 18:43:06.038268 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 31944
2026-09-27T18:43:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1149 ms)
[----------] 6 tests from EncryptionPolicyTest (9687 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:06 dist-test-slave-z47j krb5kdc[32468](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:06 dist-test-slave-z47j krb5kdc[32468](info): set up 2 sockets
Sep 27 18:43:06 dist-test-slave-z47j krb5kdc[32468](info): commencing operation
krb5kdc: starting...
W20260927 18:43:07.102041 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:07 dist-test-slave-z47j krb5kdc[32468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:43:07.230515 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38997
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:07.319339 32481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:07.319602 32481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:07.319640 32481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:07.320824 32481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:07.320894 32481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:07.320919 32481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:07.320947 32481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:07.320978 32481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:07.322932 32481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38997
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:07.323246 32481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:07.323498 32481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:07.328852 32487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:07.328900 32489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:07.328995 32481 server_base.cc:1034] running on GCE node
W20260927 18:43:07.328926 32486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:07.329509 32481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:07.329932 32481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:07.331121 32481 hybrid_clock.cc:648] HybridClock initialized: now 1790534587331093 us; error 46 us; skew 500 ppm
Sep 27 18:43:07 dist-test-slave-z47j krb5kdc[32468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534587, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:07.333887 32481 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:43:07.334390 32481 webserver.cc:460] Webserver started at http://127.0.0.1:34331/ using document root <none> and password file <none>
I20260927 18:43:07.334645 32481 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:07.334718 32481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:07.334882 32481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:07.336081 32481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "583efc6b1c8a47dc82e0e18a3cc1fa2e"
format_stamp: "Formatted at 2026-09-27 18:43:07 on dist-test-slave-z47j"
server_key: "c085f399b7b12d5fb316cba5c11862b5"
server_key_iv: "31a8b21ffecbefc2b1502dbe1129902e"
server_key_version: "encryptionkey@0"
I20260927 18:43:07.336460 32481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "583efc6b1c8a47dc82e0e18a3cc1fa2e"
format_stamp: "Formatted at 2026-09-27 18:43:07 on dist-test-slave-z47j"
server_key: "c085f399b7b12d5fb316cba5c11862b5"
server_key_iv: "31a8b21ffecbefc2b1502dbe1129902e"
server_key_version: "encryptionkey@0"
I20260927 18:43:07.337911 32481 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:43:07.338752 32496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:07.338961 32481 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:43:07.339080 32481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "583efc6b1c8a47dc82e0e18a3cc1fa2e"
format_stamp: "Formatted at 2026-09-27 18:43:07 on dist-test-slave-z47j"
server_key: "c085f399b7b12d5fb316cba5c11862b5"
server_key_iv: "31a8b21ffecbefc2b1502dbe1129902e"
server_key_version: "encryptionkey@0"
I20260927 18:43:07.339161 32481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:07.350044 32481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:07.352423 32481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:07.352594 32481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:07.357200 32481 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38997
I20260927 18:43:07.357316 32548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38997 every 8 connection(s)
I20260927 18:43:07.357728 32481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:07.358803 32549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:07.361508 32549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e: Bootstrap starting.
I20260927 18:43:07.362239 32549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:07.362625 32549 log.cc:824] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:07.363562 32549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e: No bootstrap required, opened a new log
I20260927 18:43:07.365366 32549 raft_consensus.cc:348] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "583efc6b1c8a47dc82e0e18a3cc1fa2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38997 } }
I20260927 18:43:07.365528 32549 raft_consensus.cc:374] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:07.365552 32549 raft_consensus.cc:729] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 583efc6b1c8a47dc82e0e18a3cc1fa2e, State: Initialized, Role: FOLLOWER
I20260927 18:43:07.365772 32549 consensus_queue.cc:260] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "583efc6b1c8a47dc82e0e18a3cc1fa2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38997 } }
I20260927 18:43:07.365850 32549 raft_consensus.cc:388] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:07.365875 32549 raft_consensus.cc:482] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:07.365936 32549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:07.366328 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 32481
I20260927 18:43:07.366478 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:07.366686 12271 external_mini_cluster.cc:1428] Setting key eaafd9b39d9b0775993ce18feb32489f
I20260927 18:43:07.366685 32549 raft_consensus.cc:504] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "583efc6b1c8a47dc82e0e18a3cc1fa2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38997 } }
I20260927 18:43:07.366807 32549 leader_election.cc:302] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 583efc6b1c8a47dc82e0e18a3cc1fa2e; no voters:
I20260927 18:43:07.367043 32549 leader_election.cc:288] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:07.367064 32552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:07.367197 32552 raft_consensus.cc:686] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e [term 1 LEADER]: Becoming Leader. State: Replica: 583efc6b1c8a47dc82e0e18a3cc1fa2e, State: Running, Role: LEADER
I20260927 18:43:07.367276 32552 consensus_queue.cc:237] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "583efc6b1c8a47dc82e0e18a3cc1fa2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38997 } }
I20260927 18:43:07.367341 32549 sys_catalog.cc:564] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:07.367638 32552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "583efc6b1c8a47dc82e0e18a3cc1fa2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "583efc6b1c8a47dc82e0e18a3cc1fa2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38997 } } }
I20260927 18:43:07.367853 32552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:07.368065 32552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 583efc6b1c8a47dc82e0e18a3cc1fa2e. Latest consensus state: current_term: 1 leader_uuid: "583efc6b1c8a47dc82e0e18a3cc1fa2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "583efc6b1c8a47dc82e0e18a3cc1fa2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38997 } } }
I20260927 18:43:07.368211 32552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:07.368592 32561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:07.369259 32561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:07.371347 32561 catalog_manager.cc:1347] Generated new cluster ID: a86103ff9cbe443197fd5860806f1b2e
I20260927 18:43:07.371408 32561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:43:07 dist-test-slave-z47j krb5kdc[32468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 18:43:07.377905 32561 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:07.378679 32561 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:07.379534 32556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:07.367291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34811 (local address 127.0.0.1:38997)
0927 18:43:07.367784 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:07.367795 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:07.367828 (+ 33us) server_negotiation.cc:407] Connection header received
0927 18:43:07.367979 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:07.367987 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:07.368050 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:07.368234 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:07.369142 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:07.370582 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:07.371706 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:07.372181 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:07.372322 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:07.375123 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:07.375128 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:07.375129 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:07.375142 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:07.377541 (+ 2399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:07.378104 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:07.378107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:07.378108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:07.378200 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:07.378457 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:07.378458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:07.378458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:07.378712 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:07.378910 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:07.379187 (+ 277us) server_negotiation.cc:299] Negotiation successful
0927 18:43:07.379308 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":438,"thread_start_us":139,"threads_started":1}
I20260927 18:43:07.380658 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:43:07.393124 32556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:07.385965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46265 (local address 127.0.0.1:38997)
0927 18:43:07.386018 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:07.386020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:07.386295 (+ 275us) server_negotiation.cc:407] Connection header received
0927 18:43:07.386376 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:07.386377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:07.386388 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:07.386445 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:07.387003 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:07.387596 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:07.388527 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:07.388801 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:07.388898 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:07.390352 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:07.390354 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:07.390354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:07.390362 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:07.391854 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:07.392244 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:07.392245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:07.392246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:07.392306 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:07.392550 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:07.392551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:07.392551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:07.392679 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:07.392796 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:07.392935 (+ 139us) server_negotiation.cc:299] Negotiation successful
0927 18:43:07.392998 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 18:43:07.398010 32561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 583efc6b1c8a47dc82e0e18a3cc1fa2e: Generated new TSK 0
I20260927 18:43:07.398252 32561 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:07.407776 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 32481
W20260927 18:43:07.412706 32466 connection.cc:535] client connection to 127.0.0.1:38997 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38997): Connection reset by peer (error 104)
2026-09-27T18:43:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1367 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:07 dist-test-slave-z47j krb5kdc[32580](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:07 dist-test-slave-z47j krb5kdc[32580](info): set up 2 sockets
Sep 27 18:43:07 dist-test-slave-z47j krb5kdc[32580](info): commencing operation
krb5kdc: starting...
W20260927 18:43:08.462354 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:08 dist-test-slave-z47j krb5kdc[32580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:43:08.594827 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35555
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:08.685806 32593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:08.686132 32593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:08.686173 32593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:08.687681 32593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:08.687789 32593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:08.687819 32593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:08.687855 32593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:08.687882 32593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:08.689697 32593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35555
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:08.690024 32593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:08.690296 32593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:08.695946 32599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:08.695993 32598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:08.695945 32601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:08.696271 32593 server_base.cc:1034] running on GCE node
I20260927 18:43:08.696810 32593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:08.697201 32593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:08.698407 32593 hybrid_clock.cc:648] HybridClock initialized: now 1790534588698410 us; error 152 us; skew 500 ppm
Sep 27 18:43:08 dist-test-slave-z47j krb5kdc[32580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534588, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:08.702030 32593 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:43:08.702500 32593 webserver.cc:460] Webserver started at http://127.0.0.1:35385/ using document root <none> and password file <none>
I20260927 18:43:08.702773 32593 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:08.702824 32593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:08.702970 32593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:08.704113 32593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "5a037269d40d43bd884ac89a4ba37463"
format_stamp: "Formatted at 2026-09-27 18:43:08 on dist-test-slave-z47j"
server_key: "1d1cd556da1d36bca2948b11df63ac25"
server_key_iv: "197401ad23260aef435b6018f0453493"
server_key_version: "encryptionkey@0"
I20260927 18:43:08.704496 32593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "5a037269d40d43bd884ac89a4ba37463"
format_stamp: "Formatted at 2026-09-27 18:43:08 on dist-test-slave-z47j"
server_key: "1d1cd556da1d36bca2948b11df63ac25"
server_key_iv: "197401ad23260aef435b6018f0453493"
server_key_version: "encryptionkey@0"
I20260927 18:43:08.706173 32593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:43:08.707017 32608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:08.707280 32593 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:08.707401 32593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "5a037269d40d43bd884ac89a4ba37463"
format_stamp: "Formatted at 2026-09-27 18:43:08 on dist-test-slave-z47j"
server_key: "1d1cd556da1d36bca2948b11df63ac25"
server_key_iv: "197401ad23260aef435b6018f0453493"
server_key_version: "encryptionkey@0"
I20260927 18:43:08.707486 32593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:08.718848 32593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:08.720971 32593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:08.721134 32593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:08.725543 32593 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35555
I20260927 18:43:08.725683 32660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35555 every 8 connection(s)
I20260927 18:43:08.726025 32593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:08.727142 32661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:08.730090 32661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463: Bootstrap starting.
I20260927 18:43:08.730494 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 32593
I20260927 18:43:08.730602 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:08.730850 12271 external_mini_cluster.cc:1428] Setting key 3736ff7cf0371c9688bea13bf549860f
I20260927 18:43:08.730880 32661 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:08.731304 32661 log.cc:824] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:08.732345 32661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463: No bootstrap required, opened a new log
I20260927 18:43:08.734948 32661 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a037269d40d43bd884ac89a4ba37463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35555 } }
I20260927 18:43:08.735129 32661 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:08.735179 32661 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a037269d40d43bd884ac89a4ba37463, State: Initialized, Role: FOLLOWER
I20260927 18:43:08.735360 32661 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a037269d40d43bd884ac89a4ba37463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35555 } }
I20260927 18:43:08.735468 32661 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:08.735513 32661 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:08.735565 32661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:08.736263 32661 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a037269d40d43bd884ac89a4ba37463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35555 } }
I20260927 18:43:08.736385 32661 leader_election.cc:302] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a037269d40d43bd884ac89a4ba37463; no voters:
I20260927 18:43:08.736600 32661 leader_election.cc:288] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:08.736673 32666 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:08.736887 32666 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463 [term 1 LEADER]: Becoming Leader. State: Replica: 5a037269d40d43bd884ac89a4ba37463, State: Running, Role: LEADER
I20260927 18:43:08.736953 32661 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:08.737121 32666 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a037269d40d43bd884ac89a4ba37463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35555 } }
I20260927 18:43:08.737601 32666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a037269d40d43bd884ac89a4ba37463. Latest consensus state: current_term: 1 leader_uuid: "5a037269d40d43bd884ac89a4ba37463" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a037269d40d43bd884ac89a4ba37463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35555 } } }
I20260927 18:43:08.737802 32666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:08.737776 32667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a037269d40d43bd884ac89a4ba37463" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a037269d40d43bd884ac89a4ba37463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35555 } } }
I20260927 18:43:08.738034 32667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:08.738101 32664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:08.731586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59953 (local address 127.0.0.1:35555)
0927 18:43:08.731839 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:08.731850 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:08.731881 (+ 31us) server_negotiation.cc:407] Connection header received
0927 18:43:08.732835 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:08.732845 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:08.732911 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:08.733100 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:08.733337 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:08.734286 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:08.735116 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:08.735751 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:08.735875 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:08.736776 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:08.736781 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:08.736781 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:08.737017 (+ 236us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:08.737060 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:08.737076 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:08.737140 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:08.737315 (+ 175us) server_negotiation.cc:299] Negotiation successful
0927 18:43:08.737346 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":84,"threads_started":1}
I20260927 18:43:08.739004 32671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:08.739380 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:43:08.739707 32671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:08.741677 32671 catalog_manager.cc:1347] Generated new cluster ID: 8adddbcc812841b5af0d6428605ccccb
I20260927 18:43:08.741758 32671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:08.745062 32664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:08.741333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45567 (local address 127.0.0.1:35555)
0927 18:43:08.741385 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:08.741388 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:08.741503 (+ 115us) server_negotiation.cc:407] Connection header received
0927 18:43:08.741630 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:08.741631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:08.741642 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:08.741671 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:08.741895 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:08.742634 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:08.743615 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:08.743944 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:08.743980 (+ 36us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:08.744822 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:08.744823 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:08.744824 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:08.744829 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:08.744842 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:08.744846 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:08.744874 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:08.744981 (+ 107us) server_negotiation.cc:299] Negotiation successful
0927 18:43:08.744991 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:43:08.768108 32671 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:08.768749 32671 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:08.778164 32671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a037269d40d43bd884ac89a4ba37463: Generated new TSK 0
I20260927 18:43:08.778426 32671 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:08.792176 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 32593
W20260927 18:43:08.797334 32578 connection.cc:535] client connection to 127.0.0.1:35555 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35555): Connection reset by peer (error 104)
2026-09-27T18:43:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1384 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:08 dist-test-slave-z47j krb5kdc[32692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:08 dist-test-slave-z47j krb5kdc[32692](info): set up 2 sockets
Sep 27 18:43:08 dist-test-slave-z47j krb5kdc[32692](info): commencing operation
krb5kdc: starting...
W20260927 18:43:09.854048 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:09 dist-test-slave-z47j krb5kdc[32692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:43:09.985792 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45689
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:10.075695 32705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:10.075932 32705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:10.075954 32705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:10.077149 32705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:10.077195 32705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:10.077209 32705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:10.077222 32705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:10.077234 32705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:10.078892 32705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45689
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:10.079175 32705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:10.079447 32705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:10.084800 32711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:10.084800 32710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:10.085230 32705 server_base.cc:1034] running on GCE node
W20260927 18:43:10.084897 32713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:10.085726 32705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:10.086091 32705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:10.087307 32705 hybrid_clock.cc:648] HybridClock initialized: now 1790534590087279 us; error 68 us; skew 500 ppm
Sep 27 18:43:10 dist-test-slave-z47j krb5kdc[32692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534590, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:10.090166 32705 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:43:10.090654 32705 webserver.cc:460] Webserver started at http://127.0.0.1:32783/ using document root <none> and password file <none>
I20260927 18:43:10.090900 32705 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:10.090970 32705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:10.091159 32705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:10.092319 32705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "bd496198f14c4069bbd6d803ad633400"
format_stamp: "Formatted at 2026-09-27 18:43:10 on dist-test-slave-z47j"
server_key: "0bf2e2d2229a9805039f72e8787ffd49"
server_key_iv: "c5111e3d16e22a3ace18b99bf4adb882"
server_key_version: "encryptionkey@0"
I20260927 18:43:10.092693 32705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "bd496198f14c4069bbd6d803ad633400"
format_stamp: "Formatted at 2026-09-27 18:43:10 on dist-test-slave-z47j"
server_key: "0bf2e2d2229a9805039f72e8787ffd49"
server_key_iv: "c5111e3d16e22a3ace18b99bf4adb882"
server_key_version: "encryptionkey@0"
I20260927 18:43:10.094252 32705 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:43:10.095077 32720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:10.095299 32705 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:43:10.095414 32705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "bd496198f14c4069bbd6d803ad633400"
format_stamp: "Formatted at 2026-09-27 18:43:10 on dist-test-slave-z47j"
server_key: "0bf2e2d2229a9805039f72e8787ffd49"
server_key_iv: "c5111e3d16e22a3ace18b99bf4adb882"
server_key_version: "encryptionkey@0"
I20260927 18:43:10.095491 32705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:10.098958 32705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:10.101240 32705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:10.101404 32705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:10.106088 32705 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45689
I20260927 18:43:10.106140 304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45689 every 8 connection(s)
I20260927 18:43:10.106509 32705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:10.107576 305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:10.110319 305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400: Bootstrap starting.
I20260927 18:43:10.110976 305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:10.111310 305 log.cc:824] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:10.111575 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 32705
I20260927 18:43:10.111735 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:10.111977 12271 external_mini_cluster.cc:1428] Setting key 21d8c8f808b0b22f29b558c25255d763
I20260927 18:43:10.112303 305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400: No bootstrap required, opened a new log
I20260927 18:43:10.114400 305 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd496198f14c4069bbd6d803ad633400" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45689 } }
I20260927 18:43:10.114573 305 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:10.114638 305 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd496198f14c4069bbd6d803ad633400, State: Initialized, Role: FOLLOWER
I20260927 18:43:10.114827 305 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd496198f14c4069bbd6d803ad633400" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45689 } }
I20260927 18:43:10.114931 305 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:10.115016 305 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:10.115100 305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:10.115958 309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:10.112621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37953 (local address 127.0.0.1:45689)
0927 18:43:10.112831 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:10.112848 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:10.112876 (+ 28us) server_negotiation.cc:407] Connection header received
0927 18:43:10.114426 (+ 1550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:10.114435 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:10.114472 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:10.114665 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:10.115437 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:10.115440 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:10.115440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:10.115444 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:10.115482 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:10.115493 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:10.115566 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:10.115666 (+ 100us) server_negotiation.cc:299] Negotiation successful
0927 18:43:10.115726 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":104,"threads_started":1}
I20260927 18:43:10.116379 305 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd496198f14c4069bbd6d803ad633400" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45689 } }
I20260927 18:43:10.116477 305 leader_election.cc:302] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd496198f14c4069bbd6d803ad633400; no voters:
I20260927 18:43:10.116677 305 leader_election.cc:288] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:10.116739 310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:10.116936 310 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400 [term 1 LEADER]: Becoming Leader. State: Replica: bd496198f14c4069bbd6d803ad633400, State: Running, Role: LEADER
I20260927 18:43:10.116963 305 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:10.117146 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:43:10.117105 310 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd496198f14c4069bbd6d803ad633400" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45689 } }
I20260927 18:43:10.118790 311 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd496198f14c4069bbd6d803ad633400. Latest consensus state: current_term: 1 leader_uuid: "bd496198f14c4069bbd6d803ad633400" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd496198f14c4069bbd6d803ad633400" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45689 } } }
I20260927 18:43:10.119117 311 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:10.119078 312 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd496198f14c4069bbd6d803ad633400" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd496198f14c4069bbd6d803ad633400" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45689 } } }
W20260927 18:43:10.119213 327 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:43:10.119365 327 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:43:10.119481 332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:10.119501 312 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:10.119813 332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:10.120441 309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:10.118960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40695 (local address 127.0.0.1:45689)
0927 18:43:10.119001 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:10.119003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:10.119268 (+ 265us) server_negotiation.cc:407] Connection header received
0927 18:43:10.119379 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:10.119380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:10.119386 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:10.119457 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:10.120180 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:10.120181 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:10.120182 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:10.120187 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:10.120209 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:10.120215 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:10.120287 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:10.120359 (+ 72us) server_negotiation.cc:299] Negotiation successful
0927 18:43:10.120375 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:43:10.121909 332 catalog_manager.cc:1347] Generated new cluster ID: af67e1dacdc14cb38d039f7e229bb6fd
I20260927 18:43:10.121985 332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:10.129108 332 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:10.129305 332 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:10.133973 332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd496198f14c4069bbd6d803ad633400: Generated new TSK 0
I20260927 18:43:10.134166 332 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:10.136978 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 32705
2026-09-27T18:43:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1344 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:10 dist-test-slave-z47j krb5kdc[336](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:10 dist-test-slave-z47j krb5kdc[336](info): set up 2 sockets
Sep 27 18:43:10 dist-test-slave-z47j krb5kdc[336](info): commencing operation
krb5kdc: starting...
W20260927 18:43:11.185932 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:11 dist-test-slave-z47j krb5kdc[336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:43:11.317126 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32779
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:11.407243 349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:11.407486 349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:11.407510 349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:11.408787 349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:11.408834 349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:11.408849 349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:11.408926 349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:11.408962 349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:11.410616 349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:11.410905 349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:11.411181 349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:43:11.416777 349 server_base.cc:1034] running on GCE node
W20260927 18:43:11.416726 357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:11.416798 354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:11.416726 355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:11.417272 349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:11.417662 349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:11.418869 349 hybrid_clock.cc:648] HybridClock initialized: now 1790534591418842 us; error 41 us; skew 500 ppm
Sep 27 18:43:11 dist-test-slave-z47j krb5kdc[336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534591, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:11.421830 349 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:43:11.422402 349 webserver.cc:460] Webserver started at http://127.0.0.1:40365/ using document root <none> and password file <none>
I20260927 18:43:11.422677 349 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:11.422751 349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:11.422923 349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:11.424168 349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "73b16e07c1fe4c03baefb80d5363f72b"
format_stamp: "Formatted at 2026-09-27 18:43:11 on dist-test-slave-z47j"
server_key: "682635d03d4bc1aeddc1b3ce9a55039a"
server_key_iv: "5369fb78bf889f50ce1aa355de6ee98d"
server_key_version: "encryptionkey@0"
I20260927 18:43:11.424589 349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "73b16e07c1fe4c03baefb80d5363f72b"
format_stamp: "Formatted at 2026-09-27 18:43:11 on dist-test-slave-z47j"
server_key: "682635d03d4bc1aeddc1b3ce9a55039a"
server_key_iv: "5369fb78bf889f50ce1aa355de6ee98d"
server_key_version: "encryptionkey@0"
I20260927 18:43:11.426620 349 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260927 18:43:11.427525 364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:11.427772 349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:11.427883 349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "73b16e07c1fe4c03baefb80d5363f72b"
format_stamp: "Formatted at 2026-09-27 18:43:11 on dist-test-slave-z47j"
server_key: "682635d03d4bc1aeddc1b3ce9a55039a"
server_key_iv: "5369fb78bf889f50ce1aa355de6ee98d"
server_key_version: "encryptionkey@0"
I20260927 18:43:11.427978 349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:11.452947 349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:11.455469 349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:11.455752 349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:11.461294 349 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32779
I20260927 18:43:11.461315 416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32779 every 8 connection(s)
I20260927 18:43:11.461815 349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:11.462949 417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:11.463061 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 349
I20260927 18:43:11.463187 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:11.463435 12271 external_mini_cluster.cc:1428] Setting key 420c1ffa1761eb84f7eb99e4b07f29b0
I20260927 18:43:11.466728 417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b: Bootstrap starting.
I20260927 18:43:11.467434 417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:11.467749 417 log.cc:824] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:11.468617 417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b: No bootstrap required, opened a new log
Sep 27 18:43:11 dist-test-slave-z47j krb5kdc[336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 18:43:11.471366 417 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b16e07c1fe4c03baefb80d5363f72b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32779 } }
I20260927 18:43:11.471534 417 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:11.471596 417 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73b16e07c1fe4c03baefb80d5363f72b, State: Initialized, Role: FOLLOWER
I20260927 18:43:11.471776 417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b16e07c1fe4c03baefb80d5363f72b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32779 } }
I20260927 18:43:11.471879 417 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:11.471913 417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:11.471975 417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:11.472703 417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b16e07c1fe4c03baefb80d5363f72b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32779 } }
I20260927 18:43:11.472818 417 leader_election.cc:302] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73b16e07c1fe4c03baefb80d5363f72b; no voters:
I20260927 18:43:11.473011 417 leader_election.cc:288] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:11.473165 422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:11.473409 422 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b [term 1 LEADER]: Becoming Leader. State: Replica: 73b16e07c1fe4c03baefb80d5363f72b, State: Running, Role: LEADER
I20260927 18:43:11.473444 417 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:11.473534 422 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b16e07c1fe4c03baefb80d5363f72b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32779 } }
I20260927 18:43:11.473953 422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73b16e07c1fe4c03baefb80d5363f72b. Latest consensus state: current_term: 1 leader_uuid: "73b16e07c1fe4c03baefb80d5363f72b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b16e07c1fe4c03baefb80d5363f72b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32779 } } }
I20260927 18:43:11.474058 422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:11.473968 424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73b16e07c1fe4c03baefb80d5363f72b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b16e07c1fe4c03baefb80d5363f72b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32779 } } }
I20260927 18:43:11.474149 424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:11.474304 430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:11.474946 430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:11.476598 420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:11.464047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41799 (local address 127.0.0.1:32779)
0927 18:43:11.464262 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:11.464278 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:11.464330 (+ 52us) server_negotiation.cc:407] Connection header received
0927 18:43:11.464944 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:11.464953 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:11.465009 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:11.465144 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:11.466041 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:11.467077 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:11.468153 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:11.468682 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:11.468801 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:11.471798 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:11.471803 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:11.471804 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:11.471817 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:11.474216 (+ 2399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:11.474703 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:11.474707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:11.474707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:11.474829 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:11.475078 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:11.475079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:11.475080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:11.475610 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:11.476151 (+ 541us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:11.476228 (+ 77us) server_negotiation.cc:299] Negotiation successful
0927 18:43:11.476382 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":110,"threads_started":1}
I20260927 18:43:11.476800 430 catalog_manager.cc:1347] Generated new cluster ID: 4843757ff0f348fca9c23292d907b676
I20260927 18:43:11.476856 430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:11.477619 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:43:11.488205 420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:11.479184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55189 (local address 127.0.0.1:32779)
0927 18:43:11.479228 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:11.479230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:11.481780 (+ 2550us) server_negotiation.cc:407] Connection header received
0927 18:43:11.481856 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:11.481857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:11.481885 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:11.481936 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:11.482535 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:11.483083 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:11.483949 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:11.484198 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:11.484285 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:11.485555 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:11.485556 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:11.485557 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:11.485564 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:11.487043 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:11.487420 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:11.487422 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:11.487422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:11.487485 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:11.487683 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:11.487684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:11.487685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:11.487806 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:11.487925 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:11.488021 (+ 96us) server_negotiation.cc:299] Negotiation successful
0927 18:43:11.488097 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:43:11.521992 430 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:11.522826 430 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:11.533056 430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73b16e07c1fe4c03baefb80d5363f72b: Generated new TSK 0
I20260927 18:43:11.533308 430 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:11.540828 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 349
2026-09-27T18:43:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:11 dist-test-slave-z47j krb5kdc[448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:11 dist-test-slave-z47j krb5kdc[448](info): set up 2 sockets
Sep 27 18:43:11 dist-test-slave-z47j krb5kdc[448](info): commencing operation
krb5kdc: starting...
W20260927 18:43:12.602358 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:12 dist-test-slave-z47j krb5kdc[448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:43:12.732599 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39115
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:12.819669 461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:12.819932 461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:12.819971 461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:12.821130 461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:12.821203 461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:12.821230 461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:12.821259 461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:12.821290 461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:12.822781 461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:12.823051 461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:12.823323 461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:12.828531 469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:12.828570 467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:12.828534 466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:12.828732 461 server_base.cc:1034] running on GCE node
I20260927 18:43:12.829200 461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:12.829622 461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:12.830816 461 hybrid_clock.cc:648] HybridClock initialized: now 1790534592830790 us; error 47 us; skew 500 ppm
Sep 27 18:43:12 dist-test-slave-z47j krb5kdc[448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534592, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:12.833765 461 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:43:12.834242 461 webserver.cc:460] Webserver started at http://127.0.0.1:45059/ using document root <none> and password file <none>
I20260927 18:43:12.834511 461 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:12.834561 461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:12.834723 461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:12.835971 461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "3997eaf6b9cd4d3eab439893ca16e10c"
format_stamp: "Formatted at 2026-09-27 18:43:12 on dist-test-slave-z47j"
server_key: "30bb0d7de522f793524c7b5df0673ae1"
server_key_iv: "e42d4382345122c1b876e831515277fa"
server_key_version: "encryptionkey@0"
I20260927 18:43:12.836370 461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "3997eaf6b9cd4d3eab439893ca16e10c"
format_stamp: "Formatted at 2026-09-27 18:43:12 on dist-test-slave-z47j"
server_key: "30bb0d7de522f793524c7b5df0673ae1"
server_key_iv: "e42d4382345122c1b876e831515277fa"
server_key_version: "encryptionkey@0"
I20260927 18:43:12.838033 461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260927 18:43:12.839076 476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:12.839380 461 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:12.839479 461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "3997eaf6b9cd4d3eab439893ca16e10c"
format_stamp: "Formatted at 2026-09-27 18:43:12 on dist-test-slave-z47j"
server_key: "30bb0d7de522f793524c7b5df0673ae1"
server_key_iv: "e42d4382345122c1b876e831515277fa"
server_key_version: "encryptionkey@0"
I20260927 18:43:12.839591 461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:12.855309 461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:12.857649 461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:12.857841 461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:12.863278 461 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39115
I20260927 18:43:12.863338 528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39115 every 8 connection(s)
I20260927 18:43:12.863816 461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:12.864858 529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:12.867372 529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c: Bootstrap starting.
I20260927 18:43:12.867986 529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:12.868297 529 log.cc:824] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:12.868640 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 461
I20260927 18:43:12.868798 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:12.869032 12271 external_mini_cluster.cc:1428] Setting key 1a912757cf08ddb978665177da4d10cb
I20260927 18:43:12.869091 529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c: No bootstrap required, opened a new log
I20260927 18:43:12.871333 529 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3997eaf6b9cd4d3eab439893ca16e10c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39115 } }
I20260927 18:43:12.871485 529 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:12.871508 529 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3997eaf6b9cd4d3eab439893ca16e10c, State: Initialized, Role: FOLLOWER
I20260927 18:43:12.871654 529 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3997eaf6b9cd4d3eab439893ca16e10c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39115 } }
I20260927 18:43:12.871729 529 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:12.871752 529 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:12.871821 529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:12.872433 529 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3997eaf6b9cd4d3eab439893ca16e10c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39115 } }
I20260927 18:43:12.872521 529 leader_election.cc:302] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3997eaf6b9cd4d3eab439893ca16e10c; no voters:
I20260927 18:43:12.872807 529 leader_election.cc:288] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:12.872830 534 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:12.872939 534 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c [term 1 LEADER]: Becoming Leader. State: Replica: 3997eaf6b9cd4d3eab439893ca16e10c, State: Running, Role: LEADER
I20260927 18:43:12.873040 534 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3997eaf6b9cd4d3eab439893ca16e10c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39115 } }
I20260927 18:43:12.873077 529 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:12.873360 534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3997eaf6b9cd4d3eab439893ca16e10c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3997eaf6b9cd4d3eab439893ca16e10c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39115 } } }
I20260927 18:43:12.873512 534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:12.873757 534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3997eaf6b9cd4d3eab439893ca16e10c. Latest consensus state: current_term: 1 leader_uuid: "3997eaf6b9cd4d3eab439893ca16e10c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3997eaf6b9cd4d3eab439893ca16e10c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39115 } } }
I20260927 18:43:12.873915 534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:12.874421 540 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:12.874976 540 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:12.876433 533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:12.869688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37631 (local address 127.0.0.1:39115)
0927 18:43:12.869954 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:12.869969 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:12.870001 (+ 32us) server_negotiation.cc:407] Connection header received
0927 18:43:12.870532 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:12.870539 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:12.870592 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:12.870766 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:12.871011 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:12.872519 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:12.873716 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:12.874367 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:12.874461 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:12.875667 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:12.875672 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:12.875672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:12.875856 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:12.875893 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:12.875905 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:12.876033 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:12.876135 (+ 102us) server_negotiation.cc:299] Negotiation successful
0927 18:43:12.876225 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":120,"threads_started":1}
I20260927 18:43:12.877519 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:43:12.882258 533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:12.878965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43649 (local address 127.0.0.1:39115)
0927 18:43:12.879013 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:12.879015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:12.879176 (+ 161us) server_negotiation.cc:407] Connection header received
0927 18:43:12.879228 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:12.879229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:12.879271 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:12.879327 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:12.879498 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:12.879982 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:12.881021 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:12.881213 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:12.881279 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:12.882017 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:12.882018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:12.882018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:12.882023 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:12.882035 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:12.882039 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:12.882088 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:12.882167 (+ 79us) server_negotiation.cc:299] Negotiation successful
0927 18:43:12.882189 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 18:43:12.886056 540 catalog_manager.cc:1347] Generated new cluster ID: 41c2bbbfcad3436e9e1ef4e906128a7e
I20260927 18:43:12.886139 540 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:12.893453 540 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:12.894134 540 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:12.899422 540 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3997eaf6b9cd4d3eab439893ca16e10c: Generated new TSK 0
I20260927 18:43:12.899611 540 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:12.903884 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 461
2026-09-27T18:43:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1363 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:12 dist-test-slave-z47j krb5kdc[560](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:12 dist-test-slave-z47j krb5kdc[560](info): set up 2 sockets
Sep 27 18:43:12 dist-test-slave-z47j krb5kdc[560](info): commencing operation
krb5kdc: starting...
W20260927 18:43:13.990698 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:14 dist-test-slave-z47j krb5kdc[560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:43:14.125530 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42581
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42581
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:14.222714 573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:14.222985 573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:14.223012 573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:14.224304 573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:14.224357 573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:14.224375 573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:14.224390 573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:14.224404 573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:14.225960 573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:14.226210 573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:14.226549 573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:14.232151 578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:14.232164 581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:14.232168 579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:14.232467 573 server_base.cc:1034] running on GCE node
I20260927 18:43:14.232812 573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:14.233167 573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:14.234356 573 hybrid_clock.cc:648] HybridClock initialized: now 1790534594234338 us; error 42 us; skew 500 ppm
Sep 27 18:43:14 dist-test-slave-z47j krb5kdc[560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534594, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:14.237177 573 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:43:14.237728 573 webserver.cc:460] Webserver started at http://127.0.0.1:38471/ using document root <none> and password file <none>
I20260927 18:43:14.238032 573 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:14.238153 573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:14.238302 573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:14.239776 573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "2e1397c7897848b2b0dcca682de36840"
format_stamp: "Formatted at 2026-09-27 18:43:14 on dist-test-slave-z47j"
server_key: "073ea419e7a61fc096f81f4e2872beee"
server_key_iv: "6520ba5891db607d30160d7a128a9617"
server_key_version: "encryptionkey@0"
I20260927 18:43:14.240216 573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "2e1397c7897848b2b0dcca682de36840"
format_stamp: "Formatted at 2026-09-27 18:43:14 on dist-test-slave-z47j"
server_key: "073ea419e7a61fc096f81f4e2872beee"
server_key_iv: "6520ba5891db607d30160d7a128a9617"
server_key_version: "encryptionkey@0"
I20260927 18:43:14.241746 573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260927 18:43:14.242635 588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:14.242858 573 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:14.242964 573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "2e1397c7897848b2b0dcca682de36840"
format_stamp: "Formatted at 2026-09-27 18:43:14 on dist-test-slave-z47j"
server_key: "073ea419e7a61fc096f81f4e2872beee"
server_key_iv: "6520ba5891db607d30160d7a128a9617"
server_key_version: "encryptionkey@0"
I20260927 18:43:14.243048 573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:14.246734 573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:14.248939 573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:14.249091 573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:14.254570 573 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42581
I20260927 18:43:14.254664 640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42581 every 8 connection(s)
I20260927 18:43:14.255059 573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:14.256119 641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:14.258524 641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840: Bootstrap starting.
I20260927 18:43:14.259114 641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:14.259613 641 log.cc:824] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:14.260578 641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840: No bootstrap required, opened a new log
I20260927 18:43:14.262243 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 573
I20260927 18:43:14.262369 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:14.262681 12271 external_mini_cluster.cc:1428] Setting key 2d148e33cd8c35eabcd23564025894c4
I20260927 18:43:14.262780 641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e1397c7897848b2b0dcca682de36840" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42581 } }
I20260927 18:43:14.262966 641 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:14.263013 641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e1397c7897848b2b0dcca682de36840, State: Initialized, Role: FOLLOWER
I20260927 18:43:14.263186 641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e1397c7897848b2b0dcca682de36840" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42581 } }
I20260927 18:43:14.263293 641 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:14.263355 641 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:14.263424 641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:14.264768 641 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e1397c7897848b2b0dcca682de36840" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42581 } }
I20260927 18:43:14.264895 641 leader_election.cc:302] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e1397c7897848b2b0dcca682de36840; no voters:
I20260927 18:43:14.265101 641 leader_election.cc:288] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:14.265129 646 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:14.265340 646 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840 [term 1 LEADER]: Becoming Leader. State: Replica: 2e1397c7897848b2b0dcca682de36840, State: Running, Role: LEADER
I20260927 18:43:14.265492 641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:14.265481 646 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e1397c7897848b2b0dcca682de36840" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42581 } }
I20260927 18:43:14.265782 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:14.263393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42553 (local address 127.0.0.1:42581)
0927 18:43:14.263648 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:14.263659 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:14.263695 (+ 36us) server_negotiation.cc:407] Connection header received
0927 18:43:14.263842 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:14.263850 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:14.263890 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:14.264030 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:14.264832 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:14.264835 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:14.264836 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:14.264839 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:14.264923 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:14.264940 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:14.265037 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:14.265141 (+ 104us) server_negotiation.cc:299] Negotiation successful
0927 18:43:14.265190 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":136,"threads_started":1}
I20260927 18:43:14.266100 647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e1397c7897848b2b0dcca682de36840" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e1397c7897848b2b0dcca682de36840" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42581 } } }
I20260927 18:43:14.266120 648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e1397c7897848b2b0dcca682de36840. Latest consensus state: current_term: 1 leader_uuid: "2e1397c7897848b2b0dcca682de36840" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e1397c7897848b2b0dcca682de36840" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42581 } } }
I20260927 18:43:14.266345 648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:14.266304 647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:14.266614 655 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:14.267026 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:43:14.267244 655 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:14.268960 655 catalog_manager.cc:1347] Generated new cluster ID: 211711de8ac646bd8efcc11dad01a4a7
I20260927 18:43:14.269032 655 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:14.270418 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:14.268867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34001 (local address 127.0.0.1:42581)
0927 18:43:14.268965 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:14.268967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:14.269345 (+ 378us) server_negotiation.cc:407] Connection header received
0927 18:43:14.269375 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:14.269376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:14.269384 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:14.269438 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:14.270179 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:14.270180 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:14.270180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:14.270184 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:14.270201 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:14.270206 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:14.270278 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:14.270341 (+ 63us) server_negotiation.cc:299] Negotiation successful
0927 18:43:14.270354 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260927 18:43:14.279400 655 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:14.279587 655 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:14.286576 655 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e1397c7897848b2b0dcca682de36840: Generated new TSK 0
I20260927 18:43:14.286765 655 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:14.292387 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 573
2026-09-27T18:43:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1388 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:14 dist-test-slave-z47j krb5kdc[672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:14 dist-test-slave-z47j krb5kdc[672](info): set up 2 sockets
Sep 27 18:43:14 dist-test-slave-z47j krb5kdc[672](info): commencing operation
krb5kdc: starting...
W20260927 18:43:15.354203 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:15 dist-test-slave-z47j krb5kdc[672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:43:15.488974 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42505
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:15.579041 685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:15.579341 685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:15.579392 685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:15.580724 685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:15.580825 685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:15.580856 685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:15.580891 685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:15.580916 685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:15.582722 685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:15.583050 685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:15.583312 685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:15.588893 691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:15.588836 690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:15.588830 693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:15.589277 685 server_base.cc:1034] running on GCE node
I20260927 18:43:15.589706 685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:15.590054 685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:15.591259 685 hybrid_clock.cc:648] HybridClock initialized: now 1790534595591255 us; error 40 us; skew 500 ppm
Sep 27 18:43:15 dist-test-slave-z47j krb5kdc[672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534595, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:15.594071 685 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:43:15.594594 685 webserver.cc:460] Webserver started at http://127.0.0.1:43201/ using document root <none> and password file <none>
I20260927 18:43:15.594867 685 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:15.594920 685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:15.595063 685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:15.596192 685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "24a325bf9faa4949b40fd59d717c78f4"
format_stamp: "Formatted at 2026-09-27 18:43:15 on dist-test-slave-z47j"
server_key: "1fd9efd4f1ce97fb18ec514439f58ec9"
server_key_iv: "bfd62ea928b0281244dabf72ca3b7324"
server_key_version: "encryptionkey@0"
I20260927 18:43:15.596570 685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "24a325bf9faa4949b40fd59d717c78f4"
format_stamp: "Formatted at 2026-09-27 18:43:15 on dist-test-slave-z47j"
server_key: "1fd9efd4f1ce97fb18ec514439f58ec9"
server_key_iv: "bfd62ea928b0281244dabf72ca3b7324"
server_key_version: "encryptionkey@0"
I20260927 18:43:15.598117 685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:43:15.599104 700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:15.599408 685 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:15.599498 685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "24a325bf9faa4949b40fd59d717c78f4"
format_stamp: "Formatted at 2026-09-27 18:43:15 on dist-test-slave-z47j"
server_key: "1fd9efd4f1ce97fb18ec514439f58ec9"
server_key_iv: "bfd62ea928b0281244dabf72ca3b7324"
server_key_version: "encryptionkey@0"
I20260927 18:43:15.599639 685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:15.630613 685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:15.632905 685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:15.633065 685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:15.637727 685 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42505
I20260927 18:43:15.637817 752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42505 every 8 connection(s)
I20260927 18:43:15.638172 685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:15.639261 753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:15.641985 753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4: Bootstrap starting.
I20260927 18:43:15.642668 753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:15.642992 753 log.cc:824] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:15.643946 753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4: No bootstrap required, opened a new log
I20260927 18:43:15.645097 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 685
I20260927 18:43:15.645215 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:15.645498 12271 external_mini_cluster.cc:1428] Setting key 35f3c5fedbe4bdd132c67b6e13dfa4e3
I20260927 18:43:15.646320 753 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a325bf9faa4949b40fd59d717c78f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42505 } }
I20260927 18:43:15.646559 753 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:15.646620 753 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24a325bf9faa4949b40fd59d717c78f4, State: Initialized, Role: FOLLOWER
I20260927 18:43:15.646793 753 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a325bf9faa4949b40fd59d717c78f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42505 } }
I20260927 18:43:15.646907 753 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:15.646975 753 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:15.647049 753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:15.648280 753 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a325bf9faa4949b40fd59d717c78f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42505 } }
I20260927 18:43:15.648486 753 leader_election.cc:302] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24a325bf9faa4949b40fd59d717c78f4; no voters:
I20260927 18:43:15.648726 753 leader_election.cc:288] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:15.648761 758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:15.648919 758 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4 [term 1 LEADER]: Becoming Leader. State: Replica: 24a325bf9faa4949b40fd59d717c78f4, State: Running, Role: LEADER
I20260927 18:43:15.649083 753 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:15.649089 758 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a325bf9faa4949b40fd59d717c78f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42505 } }
I20260927 18:43:15.650434 759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24a325bf9faa4949b40fd59d717c78f4. Latest consensus state: current_term: 1 leader_uuid: "24a325bf9faa4949b40fd59d717c78f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a325bf9faa4949b40fd59d717c78f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42505 } } }
I20260927 18:43:15.650641 759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:15.650811 760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24a325bf9faa4949b40fd59d717c78f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a325bf9faa4949b40fd59d717c78f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42505 } } }
I20260927 18:43:15.650888 760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:15.650990 774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:15.651695 774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 27 18:43:15 dist-test-slave-z47j krb5kdc[672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 18:43:15.653637 774 catalog_manager.cc:1347] Generated new cluster ID: 207bda627a4042468ea85b8d2caca8ae
I20260927 18:43:15.653694 774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:15.658066 757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:15.646191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.35:40749 (local address 127.0.0.1:42505)
0927 18:43:15.646399 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:15.646408 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:15.646438 (+ 30us) server_negotiation.cc:407] Connection header received
0927 18:43:15.646566 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:15.646572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:15.646658 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:15.646874 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:15.647538 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:15.648509 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:15.649488 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:15.650165 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:15.650283 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:15.653933 (+ 3650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:15.653937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:15.653937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:15.653949 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:15.655998 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:15.656579 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:15.656582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:15.656582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:15.656718 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:15.656996 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:15.656997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:15.656997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:15.657314 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:15.657474 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:15.657716 (+ 242us) server_negotiation.cc:299] Negotiation successful
0927 18:43:15.657826 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":114,"threads_started":1}
I20260927 18:43:15.659382 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:43:15.668087 774 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:15.668746 774 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:15.669442 757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:15.660871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.35:60333 (local address 127.0.0.1:42505)
0927 18:43:15.660924 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:15.660926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:15.661027 (+ 101us) server_negotiation.cc:407] Connection header received
0927 18:43:15.661062 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:15.661063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:15.661078 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:15.661157 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:15.661785 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:15.662725 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:15.663748 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:15.664135 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:15.664220 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:15.665712 (+ 1492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:15.665714 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:15.665715 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:15.665723 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:15.667692 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:15.668261 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:15.668263 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:15.668263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:15.668339 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:15.668614 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:15.668615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:15.668616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:15.668785 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:15.668931 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:15.669228 (+ 297us) server_negotiation.cc:299] Negotiation successful
0927 18:43:15.669326 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 18:43:15.676906 774 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24a325bf9faa4949b40fd59d717c78f4: Generated new TSK 0
I20260927 18:43:15.677093 774 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:15.681504 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 685
W20260927 18:43:15.686587 670 connection.cc:535] client connection to 127.0.0.1:42505 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42505): Connection reset by peer (error 104)
2026-09-27T18:43:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1389 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:15 dist-test-slave-z47j krb5kdc[784](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:15 dist-test-slave-z47j krb5kdc[784](info): set up 2 sockets
Sep 27 18:43:15 dist-test-slave-z47j krb5kdc[784](info): commencing operation
krb5kdc: starting...
W20260927 18:43:16.786201 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:16 dist-test-slave-z47j krb5kdc[784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:43:16.924593 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42443
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:17.011132 797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:17.011385 797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:17.011423 797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:17.012571 797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:17.012642 797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:17.012668 797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:17.012689 797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:17.012719 797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:17.014259 797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:17.014557 797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:17.014796 797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:17.020183 802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:17.020139 803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:17.020239 805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:17.020530 797 server_base.cc:1034] running on GCE node
I20260927 18:43:17.020916 797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:17.021364 797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:17.022583 797 hybrid_clock.cc:648] HybridClock initialized: now 1790534597022562 us; error 69 us; skew 500 ppm
Sep 27 18:43:17 dist-test-slave-z47j krb5kdc[784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534597, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:17.025910 797 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:43:17.026515 797 webserver.cc:460] Webserver started at http://127.0.0.1:38963/ using document root <none> and password file <none>
I20260927 18:43:17.026844 797 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:17.026904 797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:17.027063 797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:17.028285 797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "f8802caf42cd438888492f01783de008"
format_stamp: "Formatted at 2026-09-27 18:43:17 on dist-test-slave-z47j"
server_key: "aa2f506528cdf9805b9f6774a8ae1472"
server_key_iv: "ea2fd4c580a40d424d66ae963aa89060"
server_key_version: "encryptionkey@0"
I20260927 18:43:17.028680 797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "f8802caf42cd438888492f01783de008"
format_stamp: "Formatted at 2026-09-27 18:43:17 on dist-test-slave-z47j"
server_key: "aa2f506528cdf9805b9f6774a8ae1472"
server_key_iv: "ea2fd4c580a40d424d66ae963aa89060"
server_key_version: "encryptionkey@0"
I20260927 18:43:17.030393 797 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260927 18:43:17.031462 812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:17.031791 797 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:17.031925 797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "f8802caf42cd438888492f01783de008"
format_stamp: "Formatted at 2026-09-27 18:43:17 on dist-test-slave-z47j"
server_key: "aa2f506528cdf9805b9f6774a8ae1472"
server_key_iv: "ea2fd4c580a40d424d66ae963aa89060"
server_key_version: "encryptionkey@0"
I20260927 18:43:17.032014 797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:17.057176 797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:17.060117 797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:17.060274 797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:17.067654 797 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42443
I20260927 18:43:17.067761 864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42443 every 8 connection(s)
I20260927 18:43:17.068183 797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:17.069335 865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:17.071430 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 797
I20260927 18:43:17.071540 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:17.071795 12271 external_mini_cluster.cc:1428] Setting key 80057a4f02e7d3aa71b54d5e82843e58
I20260927 18:43:17.072153 865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008: Bootstrap starting.
I20260927 18:43:17.072954 865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:17.073307 865 log.cc:824] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:17.074445 865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008: No bootstrap required, opened a new log
I20260927 18:43:17.077196 865 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8802caf42cd438888492f01783de008" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } }
I20260927 18:43:17.077436 865 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:17.077484 865 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8802caf42cd438888492f01783de008, State: Initialized, Role: FOLLOWER
I20260927 18:43:17.077708 865 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8802caf42cd438888492f01783de008" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } }
I20260927 18:43:17.077814 865 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:17.077858 865 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:17.077910 865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:17.078784 868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:17.072520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.35:42207 (local address 127.0.0.1:42443)
0927 18:43:17.072779 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:17.072790 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:17.072830 (+ 40us) server_negotiation.cc:407] Connection header received
0927 18:43:17.073459 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:17.073468 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:17.073539 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:17.073771 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:17.074247 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:17.075253 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:17.076511 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:17.076961 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:17.077058 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:17.077966 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:17.077970 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:17.077971 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:17.078187 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:17.078221 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:17.078233 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:17.078310 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:17.078539 (+ 229us) server_negotiation.cc:299] Negotiation successful
0927 18:43:17.078595 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":111,"threads_started":1}
I20260927 18:43:17.078796 865 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8802caf42cd438888492f01783de008" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } }
I20260927 18:43:17.078971 865 leader_election.cc:302] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8802caf42cd438888492f01783de008; no voters:
I20260927 18:43:17.079231 865 leader_election.cc:288] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:17.079351 870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:17.079540 865 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:17.079636 870 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008 [term 1 LEADER]: Becoming Leader. State: Replica: f8802caf42cd438888492f01783de008, State: Running, Role: LEADER
I20260927 18:43:17.079830 870 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8802caf42cd438888492f01783de008" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } }
I20260927 18:43:17.080307 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:43:17.081068 871 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8802caf42cd438888492f01783de008" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8802caf42cd438888492f01783de008" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } } }
I20260927 18:43:17.081171 871 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:17.081460 872 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8802caf42cd438888492f01783de008. Latest consensus state: current_term: 1 leader_uuid: "f8802caf42cd438888492f01783de008" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8802caf42cd438888492f01783de008" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } } }
I20260927 18:43:17.081564 872 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008 [sys.catalog]: This master's current role is: LEADER
W20260927 18:43:17.081764 885 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:43:17.081929 885 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:43:17.082314 886 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:17.082568 886 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:17.084784 886 catalog_manager.cc:1347] Generated new cluster ID: 9ba24f094f014147aa1f5a59b4b6c51e
I20260927 18:43:17.084856 886 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:17.086509 868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:17.082198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.35:34509 (local address 127.0.0.1:42443)
0927 18:43:17.082245 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:17.082247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:17.082436 (+ 189us) server_negotiation.cc:407] Connection header received
0927 18:43:17.082540 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:17.082541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:17.082557 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:17.082636 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:17.082843 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:17.083674 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:17.085041 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:17.085315 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:17.085354 (+ 39us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:17.086248 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:17.086250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:17.086250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:17.086255 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:17.086272 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:17.086277 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:17.086336 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:17.086413 (+ 77us) server_negotiation.cc:299] Negotiation successful
0927 18:43:17.086424 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 18:43:17.110709 886 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:17.111510 886 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:17.115402 886 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8802caf42cd438888492f01783de008: Generated new TSK 0
I20260927 18:43:17.115585 886 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:17.121822 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 797
W20260927 18:43:17.128400 782 connection.cc:535] client connection to 127.0.0.1:42443 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42443): Connection reset by peer (error 104)
2026-09-27T18:43:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:17 dist-test-slave-z47j krb5kdc[896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:17 dist-test-slave-z47j krb5kdc[896](info): set up 2 sockets
Sep 27 18:43:17 dist-test-slave-z47j krb5kdc[896](info): commencing operation
krb5kdc: starting...
W20260927 18:43:18.169972 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:18 dist-test-slave-z47j krb5kdc[896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:43:18.305471 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43479
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:18.395746 909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:18.396061 909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:18.396101 909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:18.397411 909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:18.397492 909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:18.397518 909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:18.397539 909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:18.397568 909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:18.399207 909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:18.399528 909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:18.399814 909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:18.405827 915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:18.406041 909 server_base.cc:1034] running on GCE node
W20260927 18:43:18.405808 917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:18.405827 914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:18.406476 909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:18.406867 909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:18.408053 909 hybrid_clock.cc:648] HybridClock initialized: now 1790534598408034 us; error 36 us; skew 500 ppm
Sep 27 18:43:18 dist-test-slave-z47j krb5kdc[896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534598, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:18.411118 909 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:43:18.411722 909 webserver.cc:460] Webserver started at http://127.0.0.1:38847/ using document root <none> and password file <none>
I20260927 18:43:18.411983 909 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:18.412056 909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:18.412216 909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:18.413492 909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "7e5db9bf073b49e98cb5d1da50f83891"
format_stamp: "Formatted at 2026-09-27 18:43:18 on dist-test-slave-z47j"
server_key: "727421c290751a8d9bfa25345dda6978"
server_key_iv: "aa5b3550e2c030c8a320b89b4b7dc190"
server_key_version: "encryptionkey@0"
I20260927 18:43:18.414113 909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "7e5db9bf073b49e98cb5d1da50f83891"
format_stamp: "Formatted at 2026-09-27 18:43:18 on dist-test-slave-z47j"
server_key: "727421c290751a8d9bfa25345dda6978"
server_key_iv: "aa5b3550e2c030c8a320b89b4b7dc190"
server_key_version: "encryptionkey@0"
I20260927 18:43:18.416025 909 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:43:18.417019 924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:18.417284 909 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:18.417407 909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "7e5db9bf073b49e98cb5d1da50f83891"
format_stamp: "Formatted at 2026-09-27 18:43:18 on dist-test-slave-z47j"
server_key: "727421c290751a8d9bfa25345dda6978"
server_key_iv: "aa5b3550e2c030c8a320b89b4b7dc190"
server_key_version: "encryptionkey@0"
I20260927 18:43:18.417496 909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:18.421303 909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:18.423631 909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:18.423822 909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:18.429057 909 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43479
I20260927 18:43:18.429143 976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43479 every 8 connection(s)
I20260927 18:43:18.429531 909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:18.430660 977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:18.431126 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 909
I20260927 18:43:18.431265 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:18.431486 12271 external_mini_cluster.cc:1428] Setting key 585e0be8ba5f30a7b1d00f1e77f04352
I20260927 18:43:18.434142 977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891: Bootstrap starting.
I20260927 18:43:18.434701 980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:18.432154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.35:49491 (local address 127.0.0.1:43479)
0927 18:43:18.432393 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:18.432403 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:18.432438 (+ 35us) server_negotiation.cc:407] Connection header received
0927 18:43:18.433020 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:18.433028 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:18.433057 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:18.433222 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:18.434065 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:18.434069 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:18.434069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:18.434073 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:18.434117 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:18.434130 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:18.434194 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:18.434330 (+ 136us) server_negotiation.cc:299] Negotiation successful
0927 18:43:18.434385 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":101,"threads_started":1}
I20260927 18:43:18.434914 977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:18.435240 977 log.cc:824] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:18.435917 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:43:18.435989 977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891: No bootstrap required, opened a new log
I20260927 18:43:18.438391 977 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e5db9bf073b49e98cb5d1da50f83891" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43479 } }
I20260927 18:43:18.438565 977 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:18.438602 977 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e5db9bf073b49e98cb5d1da50f83891, State: Initialized, Role: FOLLOWER
I20260927 18:43:18.438767 977 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e5db9bf073b49e98cb5d1da50f83891" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43479 } }
I20260927 18:43:18.438848 980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:18.437622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.35:50847 (local address 127.0.0.1:43479)
0927 18:43:18.437664 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:18.437666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:18.437719 (+ 53us) server_negotiation.cc:407] Connection header received
0927 18:43:18.437801 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:18.437802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:18.437808 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:18.437867 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:18.438536 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:18.438538 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:18.438538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:18.438543 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:18.438557 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:18.438562 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:18.438620 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:18.438712 (+ 92us) server_negotiation.cc:299] Negotiation successful
0927 18:43:18.438728 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 18:43:18.438910 977 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:18.438957 977 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:18.439018 977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:18.439761 977 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e5db9bf073b49e98cb5d1da50f83891" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43479 } }
I20260927 18:43:18.439896 977 leader_election.cc:302] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e5db9bf073b49e98cb5d1da50f83891; no voters:
I20260927 18:43:18.440095 977 leader_election.cc:288] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:18.440207 987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:18.440364 977 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:18.440452 987 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891 [term 1 LEADER]: Becoming Leader. State: Replica: 7e5db9bf073b49e98cb5d1da50f83891, State: Running, Role: LEADER
I20260927 18:43:18.440590 987 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e5db9bf073b49e98cb5d1da50f83891" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43479 } }
I20260927 18:43:18.441439 988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e5db9bf073b49e98cb5d1da50f83891" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e5db9bf073b49e98cb5d1da50f83891" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43479 } } }
I20260927 18:43:18.441694 988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:18.442214 989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e5db9bf073b49e98cb5d1da50f83891. Latest consensus state: current_term: 1 leader_uuid: "7e5db9bf073b49e98cb5d1da50f83891" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e5db9bf073b49e98cb5d1da50f83891" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43479 } } }
I20260927 18:43:18.442312 989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891 [sys.catalog]: This master's current role is: LEADER
W20260927 18:43:18.442392 1001 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:43:18.442454 1001 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:43:18.442546 1003 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:18.442794 1003 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:18.444764 1003 catalog_manager.cc:1347] Generated new cluster ID: 1ef5c63af87042e9bc593b66a4524e5d
I20260927 18:43:18.444828 1003 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:18.464993 1003 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:18.465267 1003 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:18.468705 1003 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e5db9bf073b49e98cb5d1da50f83891: Generated new TSK 0
I20260927 18:43:18.469022 1003 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:18.477948 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 909
2026-09-27T18:43:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1354 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:18 dist-test-slave-z47j krb5kdc[1009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:18 dist-test-slave-z47j krb5kdc[1009](info): set up 2 sockets
Sep 27 18:43:18 dist-test-slave-z47j krb5kdc[1009](info): commencing operation
krb5kdc: starting...
W20260927 18:43:19.530452 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:19 dist-test-slave-z47j krb5kdc[1009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:43:19.668619 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42269
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:19.760355 1022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:19.760643 1022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:19.760682 1022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:19.761901 1022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:19.761971 1022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:19.761996 1022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:19.762017 1022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:19.762061 1022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:19.763590 1022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42269
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:19.763887 1022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:19.764137 1022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:19.769755 1030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:19.769755 1028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:19.769759 1027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:19.769907 1022 server_base.cc:1034] running on GCE node
I20260927 18:43:19.770352 1022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:19.770735 1022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:19.771898 1022 hybrid_clock.cc:648] HybridClock initialized: now 1790534599771872 us; error 47 us; skew 500 ppm
Sep 27 18:43:19 dist-test-slave-z47j krb5kdc[1009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534599, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:19.774811 1022 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:43:19.775377 1022 webserver.cc:460] Webserver started at http://127.0.0.1:38633/ using document root <none> and password file <none>
I20260927 18:43:19.775658 1022 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:19.775728 1022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:19.775874 1022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:19.776978 1022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "655c2462f9634883b3a27491c93f0c4f"
format_stamp: "Formatted at 2026-09-27 18:43:19 on dist-test-slave-z47j"
server_key: "c4b7de3e83e9b1fcbdfef178f197bcc3"
server_key_iv: "c3e6a7cf9442c7c9269a5b225c2d2fde"
server_key_version: "encryptionkey@0"
I20260927 18:43:19.777366 1022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "655c2462f9634883b3a27491c93f0c4f"
format_stamp: "Formatted at 2026-09-27 18:43:19 on dist-test-slave-z47j"
server_key: "c4b7de3e83e9b1fcbdfef178f197bcc3"
server_key_iv: "c3e6a7cf9442c7c9269a5b225c2d2fde"
server_key_version: "encryptionkey@0"
I20260927 18:43:19.778806 1022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:43:19.779641 1037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:19.779872 1022 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:19.779978 1022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "655c2462f9634883b3a27491c93f0c4f"
format_stamp: "Formatted at 2026-09-27 18:43:19 on dist-test-slave-z47j"
server_key: "c4b7de3e83e9b1fcbdfef178f197bcc3"
server_key_iv: "c3e6a7cf9442c7c9269a5b225c2d2fde"
server_key_version: "encryptionkey@0"
I20260927 18:43:19.780066 1022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:19.794894 1022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:19.797129 1022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:19.797305 1022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:19.801570 1022 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42269
I20260927 18:43:19.801682 1089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42269 every 8 connection(s)
I20260927 18:43:19.802074 1022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:19.803112 1090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:19.804394 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 1022
I20260927 18:43:19.804536 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:19.804760 12271 external_mini_cluster.cc:1428] Setting key ee9df414a9c39bd697d4db52dbbd96e9
I20260927 18:43:19.806608 1090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f: Bootstrap starting.
I20260927 18:43:19.807529 1090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:19.807912 1090 log.cc:824] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:19.808768 1090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f: No bootstrap required, opened a new log
I20260927 18:43:19.812012 1090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "655c2462f9634883b3a27491c93f0c4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42269 } }
I20260927 18:43:19.812253 1090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:19.812309 1090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 655c2462f9634883b3a27491c93f0c4f, State: Initialized, Role: FOLLOWER
I20260927 18:43:19.812517 1090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "655c2462f9634883b3a27491c93f0c4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42269 } }
I20260927 18:43:19.812597 1090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:19.812646 1090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:19.812700 1090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f [term 0 FOLLOWER]: Advancing to term 1
Sep 27 18:43:19 dist-test-slave-z47j krb5kdc[1009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 18:43:19.813459 1090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "655c2462f9634883b3a27491c93f0c4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42269 } }
I20260927 18:43:19.813560 1090 leader_election.cc:302] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 655c2462f9634883b3a27491c93f0c4f; no voters:
I20260927 18:43:19.813861 1090 leader_election.cc:288] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:19.813899 1095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:19.814024 1095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f [term 1 LEADER]: Becoming Leader. State: Replica: 655c2462f9634883b3a27491c93f0c4f, State: Running, Role: LEADER
I20260927 18:43:19.814191 1090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:19.814173 1095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "655c2462f9634883b3a27491c93f0c4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42269 } }
I20260927 18:43:19.814670 1097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "655c2462f9634883b3a27491c93f0c4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "655c2462f9634883b3a27491c93f0c4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42269 } } }
I20260927 18:43:19.814862 1097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:19.815115 1096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 655c2462f9634883b3a27491c93f0c4f. Latest consensus state: current_term: 1 leader_uuid: "655c2462f9634883b3a27491c93f0c4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "655c2462f9634883b3a27491c93f0c4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42269 } } }
I20260927 18:43:19.815508 1096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:19.815856 1103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:19.816524 1103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:19.818346 1093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:19.805352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.35:46843 (local address 127.0.0.1:42269)
0927 18:43:19.805531 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:19.805539 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:19.805568 (+ 29us) server_negotiation.cc:407] Connection header received
0927 18:43:19.806391 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:19.806398 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:19.806450 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:19.806576 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:19.807556 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:19.808602 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:19.809776 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:19.810235 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:19.810331 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:19.813720 (+ 3389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:19.813724 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:19.813724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:19.813736 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:19.815858 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:19.816377 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:19.816381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:19.816381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:19.816495 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:19.816737 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:19.816739 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:19.816739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:19.817238 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:19.817934 (+ 696us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:19.818071 (+ 137us) server_negotiation.cc:299] Negotiation successful
0927 18:43:19.818181 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
I20260927 18:43:19.818645 1103 catalog_manager.cc:1347] Generated new cluster ID: 4520a6f064e94a34b8b17f8e8028fc33
I20260927 18:43:19.818710 1103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:19.819473 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:43:19.831820 1093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:19.821990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.35:57491 (local address 127.0.0.1:42269)
0927 18:43:19.822028 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:19.822030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:19.825766 (+ 3736us) server_negotiation.cc:407] Connection header received
0927 18:43:19.825840 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:19.825841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:19.825850 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:19.825920 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:19.826448 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:19.826951 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:19.827684 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:19.827896 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:19.827983 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:19.829133 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:19.829134 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:19.829134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:19.829141 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:19.830581 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:19.830963 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:19.830965 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:19.830965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:19.831034 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:19.831258 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:19.831259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:19.831259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:19.831374 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:19.831484 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:19.831604 (+ 120us) server_negotiation.cc:299] Negotiation successful
0927 18:43:19.831701 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 18:43:19.845659 1103 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:19.846442 1103 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:19.852973 1103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 655c2462f9634883b3a27491c93f0c4f: Generated new TSK 0
I20260927 18:43:19.853282 1103 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:19.856742 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 1022
W20260927 18:43:19.861845 1007 connection.cc:535] client connection to 127.0.0.1:42269 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42269): Connection reset by peer (error 104)
2026-09-27T18:43:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1378 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:19 dist-test-slave-z47j krb5kdc[1121](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:19 dist-test-slave-z47j krb5kdc[1121](info): set up 2 sockets
Sep 27 18:43:19 dist-test-slave-z47j krb5kdc[1121](info): commencing operation
krb5kdc: starting...
W20260927 18:43:20.914192 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:20 dist-test-slave-z47j krb5kdc[1121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:43:21.044900 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44065
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:21.135255 1134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:21.135521 1134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:21.135547 1134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:21.136729 1134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:21.136770 1134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:21.136785 1134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:21.136797 1134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:21.136809 1134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:21.138326 1134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44065
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:21.138561 1134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:21.138825 1134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:21.144604 1140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:21.144665 1134 server_base.cc:1034] running on GCE node
W20260927 18:43:21.144618 1142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:21.144659 1139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:21.145242 1134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:21.145651 1134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:21.146827 1134 hybrid_clock.cc:648] HybridClock initialized: now 1790534601146798 us; error 52 us; skew 500 ppm
Sep 27 18:43:21 dist-test-slave-z47j krb5kdc[1121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534601, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:21.149682 1134 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:43:21.150221 1134 webserver.cc:460] Webserver started at http://127.0.0.1:40037/ using document root <none> and password file <none>
I20260927 18:43:21.150514 1134 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:21.150595 1134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:21.150758 1134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:21.151974 1134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "c79fe72ffd19408c9f451e6f717a3db4"
format_stamp: "Formatted at 2026-09-27 18:43:21 on dist-test-slave-z47j"
server_key: "dc74652d13f0d367b6f49a19dab5945b"
server_key_iv: "e7b4228f638a065cbffca19c661c9d68"
server_key_version: "encryptionkey@0"
I20260927 18:43:21.152359 1134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "c79fe72ffd19408c9f451e6f717a3db4"
format_stamp: "Formatted at 2026-09-27 18:43:21 on dist-test-slave-z47j"
server_key: "dc74652d13f0d367b6f49a19dab5945b"
server_key_iv: "e7b4228f638a065cbffca19c661c9d68"
server_key_version: "encryptionkey@0"
I20260927 18:43:21.154026 1134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260927 18:43:21.155009 1149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:21.155313 1134 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:21.155473 1134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "c79fe72ffd19408c9f451e6f717a3db4"
format_stamp: "Formatted at 2026-09-27 18:43:21 on dist-test-slave-z47j"
server_key: "dc74652d13f0d367b6f49a19dab5945b"
server_key_iv: "e7b4228f638a065cbffca19c661c9d68"
server_key_version: "encryptionkey@0"
I20260927 18:43:21.155557 1134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:21.166821 1134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:21.169250 1134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:21.169514 1134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:21.175767 1134 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44065
I20260927 18:43:21.175861 1201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44065 every 8 connection(s)
I20260927 18:43:21.176329 1134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:21.177424 1202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:21.179950 1202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4: Bootstrap starting.
I20260927 18:43:21.180576 1202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:21.180938 1202 log.cc:824] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:21.180972 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 1134
I20260927 18:43:21.181087 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:21.181321 12271 external_mini_cluster.cc:1428] Setting key f65e4f0739daf94d9cdeb033f09fbe71
I20260927 18:43:21.182029 1202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4: No bootstrap required, opened a new log
I20260927 18:43:21.183764 1202 raft_consensus.cc:348] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c79fe72ffd19408c9f451e6f717a3db4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44065 } }
I20260927 18:43:21.183911 1202 raft_consensus.cc:374] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:21.183934 1202 raft_consensus.cc:729] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c79fe72ffd19408c9f451e6f717a3db4, State: Initialized, Role: FOLLOWER
I20260927 18:43:21.184204 1202 consensus_queue.cc:260] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c79fe72ffd19408c9f451e6f717a3db4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44065 } }
I20260927 18:43:21.184320 1202 raft_consensus.cc:388] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:21.184371 1202 raft_consensus.cc:482] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:21.184428 1202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:21.185024 1202 raft_consensus.cc:504] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c79fe72ffd19408c9f451e6f717a3db4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44065 } }
I20260927 18:43:21.185114 1202 leader_election.cc:302] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c79fe72ffd19408c9f451e6f717a3db4; no voters:
I20260927 18:43:21.185319 1202 leader_election.cc:288] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:21.185372 1207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:21.185544 1207 raft_consensus.cc:686] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4 [term 1 LEADER]: Becoming Leader. State: Replica: c79fe72ffd19408c9f451e6f717a3db4, State: Running, Role: LEADER
I20260927 18:43:21.185643 1202 sys_catalog.cc:564] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:21.185670 1207 consensus_queue.cc:237] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c79fe72ffd19408c9f451e6f717a3db4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44065 } }
I20260927 18:43:21.186043 1207 sys_catalog.cc:455] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c79fe72ffd19408c9f451e6f717a3db4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c79fe72ffd19408c9f451e6f717a3db4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44065 } } }
I20260927 18:43:21.186187 1207 sys_catalog.cc:458] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:21.186406 1207 sys_catalog.cc:455] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c79fe72ffd19408c9f451e6f717a3db4. Latest consensus state: current_term: 1 leader_uuid: "c79fe72ffd19408c9f451e6f717a3db4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c79fe72ffd19408c9f451e6f717a3db4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44065 } } }
I20260927 18:43:21.186542 1207 sys_catalog.cc:458] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:21.186883 1215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:21.187518 1215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:21.189465 1215 catalog_manager.cc:1347] Generated new cluster ID: 03f20aad6ba0431dbb7d9fdc609d02d3
I20260927 18:43:21.189527 1215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:21.193274 1206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:21.182008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.35:42215 (local address 127.0.0.1:44065)
0927 18:43:21.182221 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:21.182240 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:21.182269 (+ 29us) server_negotiation.cc:407] Connection header received
0927 18:43:21.183748 (+ 1479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:21.183756 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:21.183814 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:21.184012 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:21.184297 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:21.186111 (+ 1814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:21.187328 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:21.187924 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:21.188056 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:21.189261 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:21.189265 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:21.189265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:21.192339 (+ 3074us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:21.192418 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:21.192437 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:21.192592 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:21.192868 (+ 276us) server_negotiation.cc:299] Negotiation successful
0927 18:43:21.192928 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":84,"threads_started":1}
I20260927 18:43:21.197804 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:43:21.204286 1215 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:21.204818 1206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:21.200400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.35:37139 (local address 127.0.0.1:44065)
0927 18:43:21.200444 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:21.200446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:21.200945 (+ 499us) server_negotiation.cc:407] Connection header received
0927 18:43:21.201059 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:21.201060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:21.201073 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:21.201103 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:21.201305 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:21.202110 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:21.203155 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:21.203494 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:21.203536 (+ 42us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:21.204517 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:21.204519 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:21.204519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:21.204524 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:21.204540 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:21.204544 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:21.204627 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:21.204697 (+ 70us) server_negotiation.cc:299] Negotiation successful
0927 18:43:21.204712 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:43:21.205163 1215 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:21.223564 1215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c79fe72ffd19408c9f451e6f717a3db4: Generated new TSK 0
I20260927 18:43:21.223898 1215 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:21.233779 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 1134
W20260927 18:43:21.238660 1119 connection.cc:535] client connection to 127.0.0.1:44065 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44065): Connection reset by peer (error 104)
2026-09-27T18:43:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1376 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:21 dist-test-slave-z47j krb5kdc[1233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:21 dist-test-slave-z47j krb5kdc[1233](info): set up 2 sockets
Sep 27 18:43:21 dist-test-slave-z47j krb5kdc[1233](info): commencing operation
krb5kdc: starting...
W20260927 18:43:22.274039 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:22 dist-test-slave-z47j krb5kdc[1233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:43:22.404749 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37113
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:22.494280 1246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:22.494544 1246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:22.494570 1246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:22.495707 1246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:22.495746 1246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:22.495760 1246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:22.495774 1246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:22.495785 1246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:22.497231 1246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37113
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:22.497463 1246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:22.497694 1246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:22.503407 1251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:22.503396 1254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:22.503407 1252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:22.503463 1246 server_base.cc:1034] running on GCE node
I20260927 18:43:22.504041 1246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:22.504405 1246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:22.505561 1246 hybrid_clock.cc:648] HybridClock initialized: now 1790534602505547 us; error 38 us; skew 500 ppm
Sep 27 18:43:22 dist-test-slave-z47j krb5kdc[1233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534602, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:22.512941 1246 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:43:22.513424 1246 webserver.cc:460] Webserver started at http://127.0.0.1:35057/ using document root <none> and password file <none>
I20260927 18:43:22.513799 1246 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:22.513872 1246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:22.514029 1246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:22.515159 1246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "eb3a0de69e7c48f491951a141a5e8e5f"
format_stamp: "Formatted at 2026-09-27 18:43:22 on dist-test-slave-z47j"
server_key: "908270434af105321e5517844a8c3864"
server_key_iv: "8a497a53287c0ecf2bc6aee62c410bd4"
server_key_version: "encryptionkey@0"
I20260927 18:43:22.515518 1246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "eb3a0de69e7c48f491951a141a5e8e5f"
format_stamp: "Formatted at 2026-09-27 18:43:22 on dist-test-slave-z47j"
server_key: "908270434af105321e5517844a8c3864"
server_key_iv: "8a497a53287c0ecf2bc6aee62c410bd4"
server_key_version: "encryptionkey@0"
I20260927 18:43:22.517019 1246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 18:43:22.517977 1261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:22.518244 1246 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:22.518354 1246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "eb3a0de69e7c48f491951a141a5e8e5f"
format_stamp: "Formatted at 2026-09-27 18:43:22 on dist-test-slave-z47j"
server_key: "908270434af105321e5517844a8c3864"
server_key_iv: "8a497a53287c0ecf2bc6aee62c410bd4"
server_key_version: "encryptionkey@0"
I20260927 18:43:22.518435 1246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:22.521850 1246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:22.524085 1246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:22.524259 1246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:22.528895 1246 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37113
I20260927 18:43:22.528959 1313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37113 every 8 connection(s)
I20260927 18:43:22.529320 1246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:22.530181 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 1246
I20260927 18:43:22.530310 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:22.530565 12271 external_mini_cluster.cc:1428] Setting key baa85a6960db2f18347f3dae60a6124e
I20260927 18:43:22.530593 1314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:22.533334 1317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:22.531199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.35:50927 (local address 127.0.0.1:37113)
0927 18:43:22.531411 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:22.531419 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:22.531446 (+ 27us) server_negotiation.cc:407] Connection header received
0927 18:43:22.532018 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:22.532025 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:22.532049 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:22.532175 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:22.532876 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:22.532879 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:22.532880 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:22.532883 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:22.532919 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:22.532929 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:22.532989 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:22.533074 (+ 85us) server_negotiation.cc:299] Negotiation successful
0927 18:43:22.533108 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":90,"threads_started":1}
I20260927 18:43:22.534437 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:43:22.534758 1314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f: Bootstrap starting.
I20260927 18:43:22.535590 1314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:22.535975 1314 log.cc:824] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:22.536909 1314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f: No bootstrap required, opened a new log
I20260927 18:43:22.537596 1317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:22.536358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.35:34779 (local address 127.0.0.1:37113)
0927 18:43:22.536426 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:22.536429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:22.536569 (+ 140us) server_negotiation.cc:407] Connection header received
0927 18:43:22.536620 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:22.536621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:22.536626 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:22.536697 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:22.537346 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:22.537347 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:22.537347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:22.537351 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:22.537366 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:22.537370 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:22.537450 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:22.537505 (+ 55us) server_negotiation.cc:299] Negotiation successful
0927 18:43:22.537519 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:43:22.539301 1314 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb3a0de69e7c48f491951a141a5e8e5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37113 } }
I20260927 18:43:22.539472 1314 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:22.539522 1314 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb3a0de69e7c48f491951a141a5e8e5f, State: Initialized, Role: FOLLOWER
I20260927 18:43:22.539670 1314 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb3a0de69e7c48f491951a141a5e8e5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37113 } }
I20260927 18:43:22.539767 1314 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:22.539811 1314 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:22.539862 1314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:22.540602 1314 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb3a0de69e7c48f491951a141a5e8e5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37113 } }
I20260927 18:43:22.540721 1314 leader_election.cc:302] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb3a0de69e7c48f491951a141a5e8e5f; no voters:
I20260927 18:43:22.540907 1314 leader_election.cc:288] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:22.541033 1324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:22.541180 1314 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:22.541250 1324 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f [term 1 LEADER]: Becoming Leader. State: Replica: eb3a0de69e7c48f491951a141a5e8e5f, State: Running, Role: LEADER
I20260927 18:43:22.541410 1324 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb3a0de69e7c48f491951a141a5e8e5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37113 } }
I20260927 18:43:22.541909 1325 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb3a0de69e7c48f491951a141a5e8e5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb3a0de69e7c48f491951a141a5e8e5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37113 } } }
I20260927 18:43:22.542032 1325 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:22.542344 1333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:22.542564 1326 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb3a0de69e7c48f491951a141a5e8e5f. Latest consensus state: current_term: 1 leader_uuid: "eb3a0de69e7c48f491951a141a5e8e5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb3a0de69e7c48f491951a141a5e8e5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37113 } } }
I20260927 18:43:22.542660 1326 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:22.543022 1333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:22.544859 1333 catalog_manager.cc:1347] Generated new cluster ID: f38c96673f7448299e02680ff656a6b0
I20260927 18:43:22.544909 1333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:22.579752 1333 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:22.580041 1333 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:22.587410 1333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb3a0de69e7c48f491951a141a5e8e5f: Generated new TSK 0
I20260927 18:43:22.587639 1333 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:22.590555 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 1246
2026-09-27T18:43:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1356 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:22 dist-test-slave-z47j krb5kdc[1345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:22 dist-test-slave-z47j krb5kdc[1345](info): set up 2 sockets
Sep 27 18:43:22 dist-test-slave-z47j krb5kdc[1345](info): commencing operation
krb5kdc: starting...
W20260927 18:43:23.630125 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:23 dist-test-slave-z47j krb5kdc[1345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:43:23.762882 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:45883
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:45883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:23.847627 1358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:23.847831 1358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:23.847853 1358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:23.849052 1358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:23.849113 1358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:23.849129 1358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:23.849141 1358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:23.849154 1358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:23.850778 1358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:45883
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:45883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:23.851022 1358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:23.851297 1358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:23.856459 1364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:23.856506 1363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:23.856724 1358 server_base.cc:1034] running on GCE node
W20260927 18:43:23.856459 1366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:23.857122 1358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:23.857486 1358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:23.858691 1358 hybrid_clock.cc:648] HybridClock initialized: now 1790534603858672 us; error 39 us; skew 500 ppm
Sep 27 18:43:23 dist-test-slave-z47j krb5kdc[1345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534603, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:23.861441 1358 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:43:23.861954 1358 webserver.cc:460] Webserver started at http://127.11.251.254:41355/ using document root <none> and password file <none>
I20260927 18:43:23.862213 1358 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:23.862283 1358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:23.862437 1358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:23.863577 1358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "9b3baa6e91c241b9839408a93b92fd32"
format_stamp: "Formatted at 2026-09-27 18:43:23 on dist-test-slave-z47j"
server_key: "3a3696846480be64410a2288b422ee74"
server_key_iv: "77c2ca3e2ce01296012d53c39befea90"
server_key_version: "encryptionkey@0"
I20260927 18:43:23.863942 1358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "9b3baa6e91c241b9839408a93b92fd32"
format_stamp: "Formatted at 2026-09-27 18:43:23 on dist-test-slave-z47j"
server_key: "3a3696846480be64410a2288b422ee74"
server_key_iv: "77c2ca3e2ce01296012d53c39befea90"
server_key_version: "encryptionkey@0"
I20260927 18:43:23.865420 1358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:43:23.866390 1373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:23.866688 1358 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:43:23.866811 1358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "9b3baa6e91c241b9839408a93b92fd32"
format_stamp: "Formatted at 2026-09-27 18:43:23 on dist-test-slave-z47j"
server_key: "3a3696846480be64410a2288b422ee74"
server_key_iv: "77c2ca3e2ce01296012d53c39befea90"
server_key_version: "encryptionkey@0"
I20260927 18:43:23.866904 1358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:23.878556 1358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:23.880517 1358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:23.880697 1358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:23.885428 1358 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:45883
I20260927 18:43:23.885547 1425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:45883 every 8 connection(s)
I20260927 18:43:23.885936 1358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:23.887065 1426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:23.888063 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 1358
I20260927 18:43:23.888234 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:23.888496 12271 external_mini_cluster.cc:1428] Setting key 101cbcae4eaa944e6b2008a29e08c45e
I20260927 18:43:23.891479 1426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32: Bootstrap starting.
I20260927 18:43:23.892204 1426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:23.892609 1426 log.cc:824] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:23.893498 1426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32: No bootstrap required, opened a new log
Sep 27 18:43:23 dist-test-slave-z47j krb5kdc[1345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:43:23.896557 1426 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b3baa6e91c241b9839408a93b92fd32" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 45883 } }
I20260927 18:43:23.896773 1426 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:23.896801 1426 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b3baa6e91c241b9839408a93b92fd32, State: Initialized, Role: FOLLOWER
I20260927 18:43:23.896965 1426 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b3baa6e91c241b9839408a93b92fd32" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 45883 } }
I20260927 18:43:23.897045 1426 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:23.897070 1426 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:23.897137 1426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:23.898044 1426 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b3baa6e91c241b9839408a93b92fd32" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 45883 } }
I20260927 18:43:23.898195 1426 leader_election.cc:302] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b3baa6e91c241b9839408a93b92fd32; no voters:
I20260927 18:43:23.898393 1426 leader_election.cc:288] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:23.898495 1431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:23.898731 1431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32 [term 1 LEADER]: Becoming Leader. State: Replica: 9b3baa6e91c241b9839408a93b92fd32, State: Running, Role: LEADER
I20260927 18:43:23.898749 1426 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:23.898924 1431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b3baa6e91c241b9839408a93b92fd32" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 45883 } }
I20260927 18:43:23.899482 1431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b3baa6e91c241b9839408a93b92fd32. Latest consensus state: current_term: 1 leader_uuid: "9b3baa6e91c241b9839408a93b92fd32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b3baa6e91c241b9839408a93b92fd32" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 45883 } } }
I20260927 18:43:23.899573 1432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b3baa6e91c241b9839408a93b92fd32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b3baa6e91c241b9839408a93b92fd32" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 45883 } } }
I20260927 18:43:23.899709 1431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:23.899744 1432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32 [sys.catalog]: This master's current role is: LEADER
W20260927 18:43:23.901234 1446 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:43:23.901317 1446 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:43:23.901388 1440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:23.901664 1440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:23.902443 1429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:23.889315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43161 (local address 127.11.251.254:45883)
0927 18:43:23.889507 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:23.889515 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:23.889546 (+ 31us) server_negotiation.cc:407] Connection header received
0927 18:43:23.890222 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:23.890229 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:23.890278 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:23.890404 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:23.891383 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:23.892452 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:23.893524 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:23.894022 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:23.894112 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:23.897307 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:23.897313 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:23.897314 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:23.897329 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:23.900406 (+ 3077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:23.901028 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:23.901031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:23.901031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:23.901139 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:23.901425 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:23.901426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:23.901426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:23.901783 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:23.901943 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:23.902120 (+ 177us) server_negotiation.cc:299] Negotiation successful
0927 18:43:23.902236 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
I20260927 18:43:23.903612 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:43:23.903641 1440 catalog_manager.cc:1347] Generated new cluster ID: e5b4f0367f1449b48949d0d2f2ada787
I20260927 18:43:23.903681 1440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:23.910176 1440 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:23.910714 1440 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:23.913393 1429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:23.905376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56619 (local address 127.11.251.254:45883)
0927 18:43:23.905425 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:23.905427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:23.905554 (+ 127us) server_negotiation.cc:407] Connection header received
0927 18:43:23.905621 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:23.905623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:23.905635 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:23.905724 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:23.906367 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:23.907182 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:23.908209 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:23.908569 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:23.908649 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:23.910247 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:23.910249 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:23.910249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:23.910257 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:23.912055 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:23.912496 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:23.912498 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:23.912498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:23.912564 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:23.912813 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:23.912814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:23.912815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:23.912973 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:23.913091 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:23.913221 (+ 130us) server_negotiation.cc:299] Negotiation successful
0927 18:43:23.913294 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 18:43:23.920522 1440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b3baa6e91c241b9839408a93b92fd32: Generated new TSK 0
I20260927 18:43:23.920711 1440 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:23.924768 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 1358
W20260927 18:43:23.929518 1343 connection.cc:535] client connection to 127.11.251.254:45883 recv error: Network error: failed to read from TLS socket (remote: 127.11.251.254:45883): Connection reset by peer (error 104)
2026-09-27T18:43:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1334 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:23 dist-test-slave-z47j krb5kdc[1457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:23 dist-test-slave-z47j krb5kdc[1457](info): set up 2 sockets
Sep 27 18:43:23 dist-test-slave-z47j krb5kdc[1457](info): commencing operation
krb5kdc: starting...
W20260927 18:43:25.006660 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:25 dist-test-slave-z47j krb5kdc[1457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:43:25.137313 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:41205
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:35155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:41205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:25.231863 1470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:25.232182 1470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:25.232224 1470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:25.233551 1470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:25.233650 1470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:25.233692 1470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:25.233729 1470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:25.233754 1470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:25.235492 1470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:35155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:41205
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:41205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:25.235808 1470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:25.236065 1470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:25.241914 1478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:25.241868 1475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:25.241961 1470 server_base.cc:1034] running on GCE node
W20260927 18:43:25.241873 1476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:25.242573 1470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:25.242976 1470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:25.244169 1470 hybrid_clock.cc:648] HybridClock initialized: now 1790534605244158 us; error 70 us; skew 500 ppm
Sep 27 18:43:25 dist-test-slave-z47j krb5kdc[1457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534605, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:25.247223 1470 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:43:25.247820 1470 webserver.cc:460] Webserver started at http://127.11.251.254:40641/ using document root <none> and password file <none>
I20260927 18:43:25.248101 1470 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:25.248178 1470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:25.248347 1470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:25.249636 1470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "f6065f70bef04bddac9262dd1611f4be"
format_stamp: "Formatted at 2026-09-27 18:43:25 on dist-test-slave-z47j"
server_key: "09e894ca3b7825f8b5c82bf957ec3671"
server_key_iv: "c2eca746f8798b4fc02152c28539274f"
server_key_version: "encryptionkey@0"
I20260927 18:43:25.250057 1470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "f6065f70bef04bddac9262dd1611f4be"
format_stamp: "Formatted at 2026-09-27 18:43:25 on dist-test-slave-z47j"
server_key: "09e894ca3b7825f8b5c82bf957ec3671"
server_key_iv: "c2eca746f8798b4fc02152c28539274f"
server_key_version: "encryptionkey@0"
I20260927 18:43:25.251607 1470 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 18:43:25.252710 1485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:25.252955 1470 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:43:25.253049 1470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "f6065f70bef04bddac9262dd1611f4be"
format_stamp: "Formatted at 2026-09-27 18:43:25 on dist-test-slave-z47j"
server_key: "09e894ca3b7825f8b5c82bf957ec3671"
server_key_iv: "c2eca746f8798b4fc02152c28539274f"
server_key_version: "encryptionkey@0"
I20260927 18:43:25.253145 1470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:25.266362 1470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:25.268693 1470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:25.268810 1470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:25.273401 1470 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:41205
I20260927 18:43:25.273506 1537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:41205 every 8 connection(s)
I20260927 18:43:25.273869 1470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:25.275049 1538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:25.277982 1538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be: Bootstrap starting.
I20260927 18:43:25.278646 1538 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:25.278913 1538 log.cc:824] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:25.279773 1538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be: No bootstrap required, opened a new log
I20260927 18:43:25.281677 1538 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6065f70bef04bddac9262dd1611f4be" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41205 } }
I20260927 18:43:25.281885 1538 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:25.281912 1538 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6065f70bef04bddac9262dd1611f4be, State: Initialized, Role: FOLLOWER
I20260927 18:43:25.282090 1538 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6065f70bef04bddac9262dd1611f4be" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41205 } }
I20260927 18:43:25.282186 1538 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:25.282212 1538 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:25.282243 1538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:25.283267 1538 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6065f70bef04bddac9262dd1611f4be" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41205 } }
I20260927 18:43:25.283362 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 1470
I20260927 18:43:25.283380 1538 leader_election.cc:302] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6065f70bef04bddac9262dd1611f4be; no voters:
I20260927 18:43:25.283536 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:25.283650 1538 leader_election.cc:288] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:25.283697 1541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:25.283844 1541 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be [term 1 LEADER]: Becoming Leader. State: Replica: f6065f70bef04bddac9262dd1611f4be, State: Running, Role: LEADER
I20260927 18:43:25.283839 12271 external_mini_cluster.cc:1428] Setting key 23c2bee011520fd29fe201d37dc61c5b
I20260927 18:43:25.283993 1538 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:25.284583 1541 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6065f70bef04bddac9262dd1611f4be" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41205 } }
I20260927 18:43:25.285387 1543 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6065f70bef04bddac9262dd1611f4be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6065f70bef04bddac9262dd1611f4be" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41205 } } }
I20260927 18:43:25.285514 1543 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:25.285699 1543 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6065f70bef04bddac9262dd1611f4be. Latest consensus state: current_term: 1 leader_uuid: "f6065f70bef04bddac9262dd1611f4be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6065f70bef04bddac9262dd1611f4be" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 41205 } } }
I20260927 18:43:25.285941 1543 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be [sys.catalog]: This master's current role is: LEADER
W20260927 18:43:25.286330 1557 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:43:25.286397 1557 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:43:25.286463 1558 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:25.286687 1558 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:25.288686 1558 catalog_manager.cc:1347] Generated new cluster ID: bf80f57e6b924889a11ef698230abb0c
I20260927 18:43:25.288743 1558 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:25.290262 1547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:25.284522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48281 (local address 127.11.251.254:41205)
0927 18:43:25.284804 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:25.284816 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:25.284865 (+ 49us) server_negotiation.cc:407] Connection header received
0927 18:43:25.285012 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:25.285019 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:25.285067 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:25.285502 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:25.285665 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:25.287063 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:25.288083 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:25.288675 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:25.289136 (+ 461us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:25.289546 (+ 410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:25.289550 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:25.289551 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:25.289778 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:25.289811 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:25.289823 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:25.289919 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:25.290000 (+ 81us) server_negotiation.cc:299] Negotiation successful
0927 18:43:25.290028 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":121,"threads_started":1}
I20260927 18:43:25.291430 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:43:25.300814 1547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:25.292810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60965 (local address 127.11.251.254:41205)
0927 18:43:25.292849 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:25.292851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:25.297843 (+ 4992us) server_negotiation.cc:407] Connection header received
0927 18:43:25.297930 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:25.297931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:25.297945 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:25.298003 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:25.298166 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:25.298703 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:25.299471 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:25.299711 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:25.299791 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:25.300519 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:25.300521 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:25.300521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:25.300526 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:25.300541 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:25.300545 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:25.300622 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:25.300689 (+ 67us) server_negotiation.cc:299] Negotiation successful
0927 18:43:25.300704 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 18:43:25.306018 1558 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:25.306650 1558 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:25.311980 1558 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6065f70bef04bddac9262dd1611f4be: Generated new TSK 0
I20260927 18:43:25.312173 1558 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:25.317150 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 1470
W20260927 18:43:25.322101 1455 connection.cc:535] client connection to 127.11.251.254:41205 recv error: Network error: failed to read from TLS socket (remote: 127.11.251.254:41205): Connection reset by peer (error 104)
2026-09-27T18:43:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1392 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:25 dist-test-slave-z47j krb5kdc[1569](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:25 dist-test-slave-z47j krb5kdc[1569](info): set up 2 sockets
Sep 27 18:43:25 dist-test-slave-z47j krb5kdc[1569](info): commencing operation
krb5kdc: starting...
W20260927 18:43:26.361927 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:26 dist-test-slave-z47j krb5kdc[1569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:43:26.498041 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:46351
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:34175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:46351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:26.583027 1582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:26.583273 1582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:26.583298 1582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:26.584525 1582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:26.584581 1582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:26.584595 1582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:26.584609 1582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:26.584621 1582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:26.586190 1582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:34175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:46351
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:46351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:26.586476 1582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:26.586727 1582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:26.592047 1590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:26.592001 1587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:26.592082 1588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:26.592517 1582 server_base.cc:1034] running on GCE node
I20260927 18:43:26.592854 1582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:26.593271 1582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:26.594472 1582 hybrid_clock.cc:648] HybridClock initialized: now 1790534606594440 us; error 65 us; skew 500 ppm
Sep 27 18:43:26 dist-test-slave-z47j krb5kdc[1569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534606, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:26.606976 1582 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:43:26.607563 1582 webserver.cc:460] Webserver started at http://127.11.251.254:35325/ using document root <none> and password file <none>
I20260927 18:43:26.607838 1582 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:26.607909 1582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:26.608086 1582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:26.609331 1582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "aa7c4dcdef9445e4a5d92d22672f0b04"
format_stamp: "Formatted at 2026-09-27 18:43:26 on dist-test-slave-z47j"
server_key: "3046a4097df9b6563b2de5d2a53e05ec"
server_key_iv: "da6931c74beed92ab14c42d57ec1f458"
server_key_version: "encryptionkey@0"
I20260927 18:43:26.609766 1582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "aa7c4dcdef9445e4a5d92d22672f0b04"
format_stamp: "Formatted at 2026-09-27 18:43:26 on dist-test-slave-z47j"
server_key: "3046a4097df9b6563b2de5d2a53e05ec"
server_key_iv: "da6931c74beed92ab14c42d57ec1f458"
server_key_version: "encryptionkey@0"
I20260927 18:43:26.611444 1582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:43:26.612460 1597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:26.612768 1582 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:26.612906 1582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "aa7c4dcdef9445e4a5d92d22672f0b04"
format_stamp: "Formatted at 2026-09-27 18:43:26 on dist-test-slave-z47j"
server_key: "3046a4097df9b6563b2de5d2a53e05ec"
server_key_iv: "da6931c74beed92ab14c42d57ec1f458"
server_key_version: "encryptionkey@0"
I20260927 18:43:26.612994 1582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:26.616812 1582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:26.619180 1582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:26.619354 1582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:26.624673 1582 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:46351
I20260927 18:43:26.624763 1649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:46351 every 8 connection(s)
I20260927 18:43:26.625167 1582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:26.626395 1650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:26.628784 1650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04: Bootstrap starting.
I20260927 18:43:26.629335 1650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:26.629674 1650 log.cc:824] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:26.630507 1650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04: No bootstrap required, opened a new log
I20260927 18:43:26.632093 1650 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7c4dcdef9445e4a5d92d22672f0b04" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46351 } }
I20260927 18:43:26.632328 1650 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:26.632366 1650 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa7c4dcdef9445e4a5d92d22672f0b04, State: Initialized, Role: FOLLOWER
I20260927 18:43:26.632506 1650 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7c4dcdef9445e4a5d92d22672f0b04" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46351 } }
I20260927 18:43:26.632601 1650 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:26.632624 1650 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:26.632699 1650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:26.633549 1650 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7c4dcdef9445e4a5d92d22672f0b04" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46351 } }
I20260927 18:43:26.633701 1650 leader_election.cc:302] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa7c4dcdef9445e4a5d92d22672f0b04; no voters:
I20260927 18:43:26.633898 1650 leader_election.cc:288] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:26.633986 1653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:26.634191 1650 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:26.634249 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 1582
I20260927 18:43:26.634218 1653 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04 [term 1 LEADER]: Becoming Leader. State: Replica: aa7c4dcdef9445e4a5d92d22672f0b04, State: Running, Role: LEADER
I20260927 18:43:26.634404 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:26.634462 1653 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7c4dcdef9445e4a5d92d22672f0b04" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46351 } }
I20260927 18:43:26.634645 12271 external_mini_cluster.cc:1428] Setting key 1a6c8e2357d39c7c1107cff88f142fc6
I20260927 18:43:26.634987 1654 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa7c4dcdef9445e4a5d92d22672f0b04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7c4dcdef9445e4a5d92d22672f0b04" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46351 } } }
I20260927 18:43:26.635093 1654 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:26.635380 1663 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:26.635546 1655 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa7c4dcdef9445e4a5d92d22672f0b04. Latest consensus state: current_term: 1 leader_uuid: "aa7c4dcdef9445e4a5d92d22672f0b04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7c4dcdef9445e4a5d92d22672f0b04" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 46351 } } }
I20260927 18:43:26.635635 1655 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:26.636196 1663 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:26.637634 1665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:26.635359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53967 (local address 127.11.251.254:46351)
0927 18:43:26.635581 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:26.635592 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:26.635868 (+ 276us) server_negotiation.cc:407] Connection header received
0927 18:43:26.636013 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:26.636020 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:26.636068 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:26.636209 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:26.637148 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:26.637152 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:26.637152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:26.637156 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:26.637209 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:26.637221 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:26.637298 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:26.637370 (+ 72us) server_negotiation.cc:299] Negotiation successful
0927 18:43:26.637415 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":84,"threads_started":1}
I20260927 18:43:26.638983 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:43:26.642030 1665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:26.640659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51685 (local address 127.11.251.254:46351)
0927 18:43:26.640717 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:26.640719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:26.640840 (+ 121us) server_negotiation.cc:407] Connection header received
0927 18:43:26.640892 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:26.640893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:26.640900 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:26.640989 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:26.641736 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:26.641739 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:26.641739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:26.641745 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:26.641764 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:26.641769 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:26.641851 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:26.641916 (+ 65us) server_negotiation.cc:299] Negotiation successful
0927 18:43:26.641934 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 18:43:26.643291 1663 catalog_manager.cc:1347] Generated new cluster ID: 44c6cbf102e64235bf8a6e1461b4a8ed
I20260927 18:43:26.643373 1663 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:26.686691 1663 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:26.686954 1663 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:26.694535 1663 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa7c4dcdef9445e4a5d92d22672f0b04: Generated new TSK 0
I20260927 18:43:26.694751 1663 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:26.702279 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 1582
2026-09-27T18:43:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1390 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:26 dist-test-slave-z47j krb5kdc[1681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:26 dist-test-slave-z47j krb5kdc[1681](info): set up 2 sockets
Sep 27 18:43:26 dist-test-slave-z47j krb5kdc[1681](info): commencing operation
krb5kdc: starting...
W20260927 18:43:27.762230 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:27 dist-test-slave-z47j krb5kdc[1681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:43:27.896543 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:34685
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:32937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:34685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:27.988448 1694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:27.988765 1694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:27.988799 1694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:27.990672 1694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:27.990738 1694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:27.990756 1694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:27.990770 1694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:27.990785 1694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:27.992789 1694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:32937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:34685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:34685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:27.993137 1694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:27.993445 1694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:43:27.999339 1694 server_base.cc:1034] running on GCE node
W20260927 18:43:27.999305 1699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:27.999287 1700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:27.999295 1702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:27.999850 1694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:28.000221 1694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:28.001402 1694 hybrid_clock.cc:648] HybridClock initialized: now 1790534608001382 us; error 38 us; skew 500 ppm
Sep 27 18:43:28 dist-test-slave-z47j krb5kdc[1681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534608, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:28.008489 1694 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:43:28.008956 1694 webserver.cc:460] Webserver started at http://127.11.251.254:42051/ using document root <none> and password file <none>
I20260927 18:43:28.009238 1694 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:28.009306 1694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:28.009456 1694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:28.010641 1694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "a472fcba013345188b1e989f75662123"
format_stamp: "Formatted at 2026-09-27 18:43:28 on dist-test-slave-z47j"
server_key: "e5216eade8682041da80b33a027eec4c"
server_key_iv: "d2f608c7b5ddee23e3df17d85d1f1389"
server_key_version: "encryptionkey@0"
I20260927 18:43:28.011018 1694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "a472fcba013345188b1e989f75662123"
format_stamp: "Formatted at 2026-09-27 18:43:28 on dist-test-slave-z47j"
server_key: "e5216eade8682041da80b33a027eec4c"
server_key_iv: "d2f608c7b5ddee23e3df17d85d1f1389"
server_key_version: "encryptionkey@0"
I20260927 18:43:28.012548 1694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:43:28.013450 1709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:28.013721 1694 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:43:28.013840 1694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "a472fcba013345188b1e989f75662123"
format_stamp: "Formatted at 2026-09-27 18:43:28 on dist-test-slave-z47j"
server_key: "e5216eade8682041da80b33a027eec4c"
server_key_iv: "d2f608c7b5ddee23e3df17d85d1f1389"
server_key_version: "encryptionkey@0"
I20260927 18:43:28.013923 1694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:28.044920 1694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:28.047546 1694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:28.047739 1694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:28.052806 1694 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:34685
I20260927 18:43:28.052899 1761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:34685 every 8 connection(s)
I20260927 18:43:28.053318 1694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:28.054389 1762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:28.056782 1762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123: Bootstrap starting.
I20260927 18:43:28.057387 1762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:28.057714 1762 log.cc:824] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:28.058653 1762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123: No bootstrap required, opened a new log
I20260927 18:43:28.060514 1762 raft_consensus.cc:348] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a472fcba013345188b1e989f75662123" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34685 } }
I20260927 18:43:28.060680 1762 raft_consensus.cc:374] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:28.060705 1762 raft_consensus.cc:729] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a472fcba013345188b1e989f75662123, State: Initialized, Role: FOLLOWER
I20260927 18:43:28.060878 1762 consensus_queue.cc:260] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a472fcba013345188b1e989f75662123" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34685 } }
I20260927 18:43:28.060956 1762 raft_consensus.cc:388] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:28.060978 1762 raft_consensus.cc:482] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:28.061030 1762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:28.062000 1762 raft_consensus.cc:504] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a472fcba013345188b1e989f75662123" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34685 } }
I20260927 18:43:28.062109 1762 leader_election.cc:302] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a472fcba013345188b1e989f75662123; no voters:
I20260927 18:43:28.062347 1762 leader_election.cc:288] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:28.062496 1765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:28.062665 1762 sys_catalog.cc:564] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:28.062764 1765 raft_consensus.cc:686] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123 [term 1 LEADER]: Becoming Leader. State: Replica: a472fcba013345188b1e989f75662123, State: Running, Role: LEADER
I20260927 18:43:28.063076 1765 consensus_queue.cc:237] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a472fcba013345188b1e989f75662123" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34685 } }
I20260927 18:43:28.063242 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 1694
I20260927 18:43:28.063375 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:28.064026 12271 external_mini_cluster.cc:1428] Setting key cf0b4487c2420a6bf0aa99102854c666
I20260927 18:43:28.064324 1767 sys_catalog.cc:455] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a472fcba013345188b1e989f75662123. Latest consensus state: current_term: 1 leader_uuid: "a472fcba013345188b1e989f75662123" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a472fcba013345188b1e989f75662123" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34685 } } }
I20260927 18:43:28.064465 1766 sys_catalog.cc:455] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a472fcba013345188b1e989f75662123" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a472fcba013345188b1e989f75662123" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 34685 } } }
I20260927 18:43:28.064651 1767 sys_catalog.cc:458] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:28.064653 1766 sys_catalog.cc:458] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123 [sys.catalog]: This master's current role is: LEADER
W20260927 18:43:28.065768 1779 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:43:28.066035 1779 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:43:28.066139 1781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:28.066354 1781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:28.069077 1781 catalog_manager.cc:1347] Generated new cluster ID: 8bbb6a1b62c2419fa3b1315f5862f95f
I20260927 18:43:28.069164 1781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:43:28 dist-test-slave-z47j krb5kdc[1681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:43:28.075770 1781 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:28.076309 1781 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:28.078240 1782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:28.064751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47177 (local address 127.11.251.254:34685)
0927 18:43:28.065099 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:28.065114 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:28.066117 (+ 1003us) server_negotiation.cc:407] Connection header received
0927 18:43:28.066296 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:28.066307 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:28.066394 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:28.066576 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:28.067470 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:28.068929 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:28.070122 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:28.070801 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:28.070920 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:28.074136 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:28.074142 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:28.074142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:28.074159 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:28.076299 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:28.076825 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:28.076827 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:28.076828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:28.076950 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:28.077186 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:28.077186 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:28.077187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:28.077507 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:28.077709 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:28.077807 (+ 98us) server_negotiation.cc:299] Negotiation successful
0927 18:43:28.077982 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":220,"threads_started":1}
I20260927 18:43:28.079437 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:43:28.081027 1781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a472fcba013345188b1e989f75662123: Generated new TSK 0
I20260927 18:43:28.081287 1781 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:28.088366 1782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:28.081080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58435 (local address 127.11.251.254:34685)
0927 18:43:28.081123 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:28.081125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:28.081380 (+ 255us) server_negotiation.cc:407] Connection header received
0927 18:43:28.081523 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:28.081525 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:28.081541 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:28.081692 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:28.082213 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:28.083061 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:28.083857 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:28.084205 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:28.084281 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:28.085600 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:28.085602 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:28.085602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:28.085618 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:28.087210 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:28.087566 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:28.087567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:28.087567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:28.087652 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:28.087863 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:28.087864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:28.087864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:28.087991 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:28.088094 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:28.088188 (+ 94us) server_negotiation.cc:299] Negotiation successful
0927 18:43:28.088272 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:43:28.090583 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 1694
2026-09-27T18:43:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1381 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:28 dist-test-slave-z47j krb5kdc[1793](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:28 dist-test-slave-z47j krb5kdc[1793](info): set up 2 sockets
Sep 27 18:43:28 dist-test-slave-z47j krb5kdc[1793](info): commencing operation
krb5kdc: starting...
W20260927 18:43:29.138507 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:29 dist-test-slave-z47j krb5kdc[1793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:43:29.277331 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:33939
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:36565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:33939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:29.361732 1806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:29.361987 1806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:29.362026 1806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:29.363351 1806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:29.363425 1806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:29.363456 1806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:29.363488 1806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:29.363514 1806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:29.365015 1806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:36565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:33939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:33939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:29.365306 1806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:29.365535 1806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:29.370759 1814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:29.370759 1811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:29.370855 1812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:29.370909 1806 server_base.cc:1034] running on GCE node
I20260927 18:43:29.371410 1806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:29.371757 1806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:29.372932 1806 hybrid_clock.cc:648] HybridClock initialized: now 1790534609372907 us; error 49 us; skew 500 ppm
Sep 27 18:43:29 dist-test-slave-z47j krb5kdc[1793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534609, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:29.375883 1806 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:43:29.376382 1806 webserver.cc:460] Webserver started at http://127.11.251.254:42877/ using document root <none> and password file <none>
I20260927 18:43:29.376641 1806 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:29.376693 1806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:29.376828 1806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:29.377961 1806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "ad46b9d0427c4d528f03a27a31fd4e5f"
format_stamp: "Formatted at 2026-09-27 18:43:29 on dist-test-slave-z47j"
server_key: "944e832d87da31f625ebafbc92e53e47"
server_key_iv: "0aa30e024ee35de5973f67bc2aa95db6"
server_key_version: "encryptionkey@0"
I20260927 18:43:29.378320 1806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "ad46b9d0427c4d528f03a27a31fd4e5f"
format_stamp: "Formatted at 2026-09-27 18:43:29 on dist-test-slave-z47j"
server_key: "944e832d87da31f625ebafbc92e53e47"
server_key_iv: "0aa30e024ee35de5973f67bc2aa95db6"
server_key_version: "encryptionkey@0"
I20260927 18:43:29.379830 1806 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:43:29.380731 1821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:29.381017 1806 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:29.381106 1806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "ad46b9d0427c4d528f03a27a31fd4e5f"
format_stamp: "Formatted at 2026-09-27 18:43:29 on dist-test-slave-z47j"
server_key: "944e832d87da31f625ebafbc92e53e47"
server_key_iv: "0aa30e024ee35de5973f67bc2aa95db6"
server_key_version: "encryptionkey@0"
I20260927 18:43:29.381206 1806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:29.389042 1806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:29.391525 1806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:29.391649 1806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:29.396543 1806 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:33939
I20260927 18:43:29.396648 1873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:33939 every 8 connection(s)
I20260927 18:43:29.396994 1806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:29.398258 1874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:29.401129 1874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f: Bootstrap starting.
I20260927 18:43:29.402136 1874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:29.402441 1874 log.cc:824] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:29.403326 1874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f: No bootstrap required, opened a new log
I20260927 18:43:29.403514 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 1806
I20260927 18:43:29.403657 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:29.403900 12271 external_mini_cluster.cc:1428] Setting key be64a907adf01bdc0fc18596b8cf146d
I20260927 18:43:29.406023 1874 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad46b9d0427c4d528f03a27a31fd4e5f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 33939 } }
I20260927 18:43:29.406250 1874 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:29.406298 1874 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad46b9d0427c4d528f03a27a31fd4e5f, State: Initialized, Role: FOLLOWER
I20260927 18:43:29.406502 1874 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad46b9d0427c4d528f03a27a31fd4e5f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 33939 } }
I20260927 18:43:29.406615 1874 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:29.406657 1874 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:29.406740 1874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:29.407564 1874 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad46b9d0427c4d528f03a27a31fd4e5f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 33939 } }
I20260927 18:43:29.407678 1874 leader_election.cc:302] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad46b9d0427c4d528f03a27a31fd4e5f; no voters:
I20260927 18:43:29.407968 1874 leader_election.cc:288] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:29.408033 1879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:29.408242 1879 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f [term 1 LEADER]: Becoming Leader. State: Replica: ad46b9d0427c4d528f03a27a31fd4e5f, State: Running, Role: LEADER
I20260927 18:43:29.408314 1874 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:29.408448 1879 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad46b9d0427c4d528f03a27a31fd4e5f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 33939 } }
I20260927 18:43:29.408943 1879 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad46b9d0427c4d528f03a27a31fd4e5f. Latest consensus state: current_term: 1 leader_uuid: "ad46b9d0427c4d528f03a27a31fd4e5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad46b9d0427c4d528f03a27a31fd4e5f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 33939 } } }
I20260927 18:43:29.409303 1880 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad46b9d0427c4d528f03a27a31fd4e5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad46b9d0427c4d528f03a27a31fd4e5f" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 33939 } } }
I20260927 18:43:29.409415 1879 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:29.409648 1880 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f [sys.catalog]: This master's current role is: LEADER
W20260927 18:43:29.410943 1894 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:43:29.411087 1894 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:43:29.411186 1889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:29.411422 1889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:29.411945 1878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:29.404520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39455 (local address 127.11.251.254:33939)
0927 18:43:29.404727 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:29.404736 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:29.404767 (+ 31us) server_negotiation.cc:407] Connection header received
0927 18:43:29.406020 (+ 1253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:29.406029 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:29.406099 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:29.406258 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:29.406733 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:29.408224 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:29.409438 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:29.410108 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:29.410592 (+ 484us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:29.411134 (+ 542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:29.411138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:29.411138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:29.411359 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:29.411396 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:29.411411 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:29.411483 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:29.411651 (+ 168us) server_negotiation.cc:299] Negotiation successful
0927 18:43:29.411721 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":100,"threads_started":1}
I20260927 18:43:29.413125 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:43:29.413635 1889 catalog_manager.cc:1347] Generated new cluster ID: 4a7657a2ed054d0b99c90235c21a88d4
I20260927 18:43:29.413686 1889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:29.419026 1878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:29.415072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35371 (local address 127.11.251.254:33939)
0927 18:43:29.415112 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:29.415113 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:29.415174 (+ 61us) server_negotiation.cc:407] Connection header received
0927 18:43:29.415267 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:29.415268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:29.415288 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:29.415352 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:29.415537 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:29.416262 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:29.417344 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:29.417726 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:29.417812 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:29.418705 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:29.418707 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:29.418708 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:29.418713 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:29.418739 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:29.418745 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:29.418822 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:29.418904 (+ 82us) server_negotiation.cc:299] Negotiation successful
0927 18:43:29.418939 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:43:29.426774 1889 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:29.427577 1889 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:29.430693 1889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad46b9d0427c4d528f03a27a31fd4e5f: Generated new TSK 0
I20260927 18:43:29.430977 1889 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:29.435035 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 1806
2026-09-27T18:43:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1350 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:29 dist-test-slave-z47j krb5kdc[1905](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:29 dist-test-slave-z47j krb5kdc[1905](info): set up 2 sockets
Sep 27 18:43:29 dist-test-slave-z47j krb5kdc[1905](info): commencing operation
krb5kdc: starting...
W20260927 18:43:30.494218 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:30 dist-test-slave-z47j krb5kdc[1905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:43:30.628774 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:36159
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:43913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:36159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:30.711207 1918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:30.711438 1918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:30.711475 1918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:30.712605 1918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:30.712674 1918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:30.712700 1918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:30.712731 1918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:30.712752 1918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:30.714334 1918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:43913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:36159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:36159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:30.714609 1918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:30.714843 1918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:30.720278 1923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:30.720299 1924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:30.720463 1918 server_base.cc:1034] running on GCE node
W20260927 18:43:30.720306 1926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:30.720885 1918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:30.721263 1918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:30.722455 1918 hybrid_clock.cc:648] HybridClock initialized: now 1790534610722399 us; error 80 us; skew 500 ppm
Sep 27 18:43:30 dist-test-slave-z47j krb5kdc[1905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534610, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:30.725312 1918 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:43:30.725939 1918 webserver.cc:460] Webserver started at http://127.11.251.254:44903/ using document root <none> and password file <none>
I20260927 18:43:30.726217 1918 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:30.726293 1918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:30.726456 1918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:30.727711 1918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "29ccccfa2b444774aa6184b6f7d5562d"
format_stamp: "Formatted at 2026-09-27 18:43:30 on dist-test-slave-z47j"
server_key: "8c79c9590f87e95115dc8878e1d33b3e"
server_key_iv: "d6132861bcac79bca593945a77c38842"
server_key_version: "encryptionkey@0"
I20260927 18:43:30.728127 1918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "29ccccfa2b444774aa6184b6f7d5562d"
format_stamp: "Formatted at 2026-09-27 18:43:30 on dist-test-slave-z47j"
server_key: "8c79c9590f87e95115dc8878e1d33b3e"
server_key_iv: "d6132861bcac79bca593945a77c38842"
server_key_version: "encryptionkey@0"
I20260927 18:43:30.729961 1918 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260927 18:43:30.730923 1933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:30.731217 1918 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:30.731341 1918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "29ccccfa2b444774aa6184b6f7d5562d"
format_stamp: "Formatted at 2026-09-27 18:43:30 on dist-test-slave-z47j"
server_key: "8c79c9590f87e95115dc8878e1d33b3e"
server_key_iv: "d6132861bcac79bca593945a77c38842"
server_key_version: "encryptionkey@0"
I20260927 18:43:30.731454 1918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:30.735430 1918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:30.738021 1918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:30.738193 1918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:30.743446 1918 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:36159
I20260927 18:43:30.743551 1985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:36159 every 8 connection(s)
I20260927 18:43:30.743918 1918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:30.744392 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 1918
I20260927 18:43:30.744544 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:30.744831 12271 external_mini_cluster.cc:1428] Setting key a653e37325adc37b3ff6a252cbf91114
I20260927 18:43:30.745325 1986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:30.748466 1989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:30.745531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36231 (local address 127.11.251.254:36159)
0927 18:43:30.745815 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:30.745825 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:30.745857 (+ 32us) server_negotiation.cc:407] Connection header received
0927 18:43:30.746972 (+ 1115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:30.746980 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:30.747015 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:30.747158 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:30.747924 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:30.747927 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:30.747927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:30.747931 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:30.747971 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:30.747982 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:30.748041 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:30.748163 (+ 122us) server_negotiation.cc:299] Negotiation successful
0927 18:43:30.748224 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":136,"threads_started":1}
I20260927 18:43:30.749193 1986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d: Bootstrap starting.
I20260927 18:43:30.749962 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:43:30.750036 1986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:30.750325 1986 log.cc:824] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:30.751235 1986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d: No bootstrap required, opened a new log
I20260927 18:43:30.753029 1989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:30.751657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34423 (local address 127.11.251.254:36159)
0927 18:43:30.751703 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:30.751705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:30.751887 (+ 182us) server_negotiation.cc:407] Connection header received
0927 18:43:30.751912 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:30.751913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:30.751918 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:30.751964 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:30.752677 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:30.752678 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:30.752679 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:30.752683 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:30.752696 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:30.752700 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:30.752760 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:30.752842 (+ 82us) server_negotiation.cc:299] Negotiation successful
0927 18:43:30.752940 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 18:43:30.753880 1986 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29ccccfa2b444774aa6184b6f7d5562d" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 36159 } }
I20260927 18:43:30.754022 1986 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:30.754045 1986 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29ccccfa2b444774aa6184b6f7d5562d, State: Initialized, Role: FOLLOWER
I20260927 18:43:30.754213 1986 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29ccccfa2b444774aa6184b6f7d5562d" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 36159 } }
I20260927 18:43:30.754292 1986 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:30.754335 1986 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:30.754390 1986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:30.754951 1986 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29ccccfa2b444774aa6184b6f7d5562d" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 36159 } }
I20260927 18:43:30.755076 1986 leader_election.cc:302] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29ccccfa2b444774aa6184b6f7d5562d; no voters:
I20260927 18:43:30.755275 1986 leader_election.cc:288] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:30.755317 1996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:30.755537 1986 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:30.755540 1996 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d [term 1 LEADER]: Becoming Leader. State: Replica: 29ccccfa2b444774aa6184b6f7d5562d, State: Running, Role: LEADER
I20260927 18:43:30.755685 1996 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29ccccfa2b444774aa6184b6f7d5562d" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 36159 } }
I20260927 18:43:30.756166 1998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29ccccfa2b444774aa6184b6f7d5562d. Latest consensus state: current_term: 1 leader_uuid: "29ccccfa2b444774aa6184b6f7d5562d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29ccccfa2b444774aa6184b6f7d5562d" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 36159 } } }
I20260927 18:43:30.756271 1998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:30.756464 1997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29ccccfa2b444774aa6184b6f7d5562d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29ccccfa2b444774aa6184b6f7d5562d" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 36159 } } }
I20260927 18:43:30.756536 1997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:30.756625 2005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:30.757191 2005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:30.759056 2005 catalog_manager.cc:1347] Generated new cluster ID: d3d55d98590e456483ce7942048d674d
I20260927 18:43:30.759119 2005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:30.765388 2005 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:30.765633 2005 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:30.781911 2005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29ccccfa2b444774aa6184b6f7d5562d: Generated new TSK 0
I20260927 18:43:30.782171 2005 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:30.786068 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 1918
2026-09-27T18:43:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1344 ms)
[----------] 18 tests from AuthTokenIssuingTest (24744 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:30 dist-test-slave-z47j krb5kdc[2017](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:30 dist-test-slave-z47j krb5kdc[2017](info): set up 2 sockets
Sep 27 18:43:30 dist-test-slave-z47j krb5kdc[2017](info): commencing operation
krb5kdc: starting...
W20260927 18:43:31.834247 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:31 dist-test-slave-z47j krb5kdc[2017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:43:31.965164 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:37689
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:35059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:37689,127.11.251.253:39819,127.11.251.252:43735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:32.054518 2030 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 18:43:32.054837 2030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:32.054872 2030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:32.054924 2030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:32.056202 2030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:32.056274 2030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:32.056310 2030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:32.056342 2030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:32.056370 2030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:32.058073 2030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:35059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:37689,127.11.251.253:39819,127.11.251.252:43735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:37689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:32.058408 2030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:32.058684 2030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:43:32.064615 2030 server_base.cc:1034] running on GCE node
W20260927 18:43:32.064574 2035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:32.064585 2038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:32.064594 2036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:32.065153 2030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:32.065519 2030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:32.066684 2030 hybrid_clock.cc:648] HybridClock initialized: now 1790534612066645 us; error 53 us; skew 500 ppm
Sep 27 18:43:32 dist-test-slave-z47j krb5kdc[2017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534612, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:32.069720 2030 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:43:32.070333 2030 webserver.cc:460] Webserver started at http://127.11.251.254:35249/ using document root <none> and password file <none>
I20260927 18:43:32.070590 2030 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:32.070657 2030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:32.070819 2030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:32.072060 2030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "2934c5e4bad94a80b2becc163faebec3"
format_stamp: "Formatted at 2026-09-27 18:43:32 on dist-test-slave-z47j"
server_key: "9763f6cc88110ad6ea82cb7f452c4dce"
server_key_iv: "7431bb0589d38ab35306c6ca3a3a8c1f"
server_key_version: "encryptionkey@0"
I20260927 18:43:32.072461 2030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "2934c5e4bad94a80b2becc163faebec3"
format_stamp: "Formatted at 2026-09-27 18:43:32 on dist-test-slave-z47j"
server_key: "9763f6cc88110ad6ea82cb7f452c4dce"
server_key_iv: "7431bb0589d38ab35306c6ca3a3a8c1f"
server_key_version: "encryptionkey@0"
I20260927 18:43:32.074033 2030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:43:32.074919 2045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:32.075160 2030 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:43:32.075266 2030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "2934c5e4bad94a80b2becc163faebec3"
format_stamp: "Formatted at 2026-09-27 18:43:32 on dist-test-slave-z47j"
server_key: "9763f6cc88110ad6ea82cb7f452c4dce"
server_key_iv: "7431bb0589d38ab35306c6ca3a3a8c1f"
server_key_version: "encryptionkey@0"
I20260927 18:43:32.075349 2030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:32.092200 2030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:32.094807 2030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:32.095000 2030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:32.100385 2030 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:37689
I20260927 18:43:32.100482 2097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:37689 every 8 connection(s)
I20260927 18:43:32.100922 2030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:32.101970 2098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:32.103767 2098 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37689 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:43:32 dist-test-slave-z47j krb5kdc[2017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534612, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:43:32.110981 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 2030
I20260927 18:43:32.111161 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:32.111393 12271 external_mini_cluster.cc:1428] Setting key bd49dce6a23b20fcc0a8e1556f0667e4
I20260927 18:43:32.115240 2101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.104783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56314 (local address 127.11.251.254:37689)
0927 18:43:32.104933 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.104938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.104981 (+ 43us) server_negotiation.cc:407] Connection header received
0927 18:43:32.105770 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.105772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.105793 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.105885 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.106543 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.107396 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.108337 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.108710 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.108806 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.111578 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:32.111587 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:32.111588 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:32.111597 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:32.113304 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.113870 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.113873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.113873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.113965 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.114242 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.114243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.114244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.114467 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:32.114535 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:32.114840 (+ 305us) server_negotiation.cc:299] Negotiation successful
0927 18:43:32.115036 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
I20260927 18:43:32.115341 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.104541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:37689 (local address 127.0.0.1:56314)
0927 18:43:32.104773 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.104835 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:32.105549 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:32.105919 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:32.105922 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:32.106239 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:32.106442 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:32.106447 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.107491 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.107492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:32.108251 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:32.108252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.108860 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.108861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:32.108949 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.109749 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:32.109759 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:32.111478 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:32.113393 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:32.113401 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:32.113404 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:32.113760 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:32.114062 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:32.114063 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:32.114063 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:32.114180 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:32.114550 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:32.114557 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:32.114646 (+ 89us) client_negotiation.cc:769] Sending connection context
0927 18:43:32.114749 (+ 103us) client_negotiation.cc:240] Negotiation successful
0927 18:43:32.114961 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
I20260927 18:43:32.116326 2098 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:43:32.116744 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.116575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:39819 (local address 127.0.0.1:44150)
0927 18:43:32.116639 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.116706 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.253:39819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260927 18:43:32.116956 2046 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.251.253:39819: connect: Connection refused (error 111)
I20260927 18:43:32.117491 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.117358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:39819 (local address 127.0.0.1:44154)
0927 18:43:32.117416 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.117458 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.253:39819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260927 18:43:32.117646 2098 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.253:39819: Network error: Client connection negotiation failed: client connection to 127.11.251.253:39819: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.11.251.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.253.keytab.
Entry for principal kudu/127.11.251.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.253.keytab.
I20260927 18:43:32.140807 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.253:39819
--webserver_interface=127.11.251.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.253.keytab
--principal=kudu/127.11.251.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:35059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:37689,127.11.251.253:39819,127.11.251.252:43735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:43:32.166970 2098 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } attempt: 1
I20260927 18:43:32.167634 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.167443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:39819 (local address 127.0.0.1:44160)
0927 18:43:32.167523 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.167592 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.253:39819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260927 18:43:32.168047 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.167949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:39819 (local address 127.0.0.1:44174)
0927 18:43:32.168000 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.168029 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.253:39819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260927 18:43:32.168192 2098 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.253:39819: Network error: Client connection negotiation failed: client connection to 127.11.251.253:39819: connect: Connection refused (error 111)
W20260927 18:43:32.231663 2105 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 18:43:32.231926 2105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:32.231945 2105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:32.231967 2105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:32.233248 2105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:32.233292 2105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:32.233306 2105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:32.233319 2105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:32.233332 2105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:32.234963 2105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:35059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:37689,127.11.251.253:39819,127.11.251.252:43735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.253.keytab
--principal=kudu/127.11.251.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.253:39819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.251.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:32.235270 2105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:32.235536 2105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:43:32.236392 2098 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } attempt: 2
I20260927 18:43:32.237000 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.236780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:39819 (local address 127.0.0.1:44176)
0927 18:43:32.236860 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.236949 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.253:39819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260927 18:43:32.237440 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.237349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:39819 (local address 127.0.0.1:44178)
0927 18:43:32.237391 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.237417 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.253:39819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260927 18:43:32.237543 2098 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.253:39819: Network error: Client connection negotiation failed: client connection to 127.11.251.253:39819: connect: Connection refused (error 111)
W20260927 18:43:32.241009 2111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:32.241024 2110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:32.241029 2113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:32.241380 2105 server_base.cc:1034] running on GCE node
I20260927 18:43:32.241688 2105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:32.242022 2105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:32.243221 2105 hybrid_clock.cc:648] HybridClock initialized: now 1790534612243197 us; error 74 us; skew 500 ppm
Sep 27 18:43:32 dist-test-slave-z47j krb5kdc[2017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534612, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:32.246186 2105 init.cc:377] Logged in from keytab as kudu/127.11.251.253@KRBTEST.COM (short username kudu)
I20260927 18:43:32.246752 2105 webserver.cc:460] Webserver started at http://127.11.251.253:44793/ using document root <none> and password file <none>
I20260927 18:43:32.246989 2105 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:32.247040 2105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:32.247157 2105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:32.248376 2105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-1/data/instance:
uuid: "33b7bca518414fb6991f864f4b20ac1c"
format_stamp: "Formatted at 2026-09-27 18:43:32 on dist-test-slave-z47j"
server_key: "6f15709633c3a3d1b9bb4f353dcfea27"
server_key_iv: "5b597074e7edd83d5222c79b9cb226b0"
server_key_version: "encryptionkey@0"
I20260927 18:43:32.248732 2105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-1/wal/instance:
uuid: "33b7bca518414fb6991f864f4b20ac1c"
format_stamp: "Formatted at 2026-09-27 18:43:32 on dist-test-slave-z47j"
server_key: "6f15709633c3a3d1b9bb4f353dcfea27"
server_key_iv: "5b597074e7edd83d5222c79b9cb226b0"
server_key_version: "encryptionkey@0"
I20260927 18:43:32.250368 2105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:43:32.251235 2120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:32.251451 2105 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:32.251549 2105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-1/wal
uuid: "33b7bca518414fb6991f864f4b20ac1c"
format_stamp: "Formatted at 2026-09-27 18:43:32 on dist-test-slave-z47j"
server_key: "6f15709633c3a3d1b9bb4f353dcfea27"
server_key_iv: "5b597074e7edd83d5222c79b9cb226b0"
server_key_version: "encryptionkey@0"
I20260927 18:43:32.251639 2105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:32.278717 2105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:32.281414 2105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:32.281661 2105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:32.286643 2105 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.253:39819
I20260927 18:43:32.286727 2172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.253:39819 every 8 connection(s)
I20260927 18:43:32.287129 2105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-1/data/info.pb
I20260927 18:43:32.288168 2173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:32.289901 2173 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37689 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:43:32 dist-test-slave-z47j krb5kdc[2017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534612, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.253@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:43:32.296996 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 2105
I20260927 18:43:32.297120 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-1/wal/instance
I20260927 18:43:32.297384 12271 external_mini_cluster.cc:1428] Setting key 453f5abc19e989fb9391651f17e5c00d
I20260927 18:43:32.300199 2101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.290891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56318 (local address 127.11.251.254:37689)
0927 18:43:32.291017 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.291019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.291237 (+ 218us) server_negotiation.cc:407] Connection header received
0927 18:43:32.291761 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.291762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.291774 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.291842 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.292471 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.292977 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.294013 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.294228 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.294315 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.296831 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:32.296833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:32.296833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:32.296840 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:32.298434 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.298943 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.298946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.298947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.299061 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.299413 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.299414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.299415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.299565 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:32.299647 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:32.300019 (+ 372us) server_negotiation.cc:299] Negotiation successful
0927 18:43:32.300119 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260927 18:43:32.300381 2175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.290849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:37689 (local address 127.0.0.1:56318)
0927 18:43:32.291088 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.291140 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:32.291656 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:32.291916 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:32.291920 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:32.292187 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:32.292381 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:32.292384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.293081 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.293082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:32.293918 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:32.293920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.294336 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.294337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:32.294454 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.295073 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:32.295082 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:32.296695 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:32.298537 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:32.298544 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:32.298547 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:32.298811 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:32.299156 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:32.299157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:32.299157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:32.299302 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:32.299654 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:32.299660 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:32.299832 (+ 172us) client_negotiation.cc:769] Sending connection context
0927 18:43:32.299975 (+ 143us) client_negotiation.cc:240] Negotiation successful
0927 18:43:32.300144 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":105,"threads_started":1}
I20260927 18:43:32.300995 2173 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:43:32 dist-test-slave-z47j krb5kdc[2017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534612, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.253@KRBTEST.COM for kudu/127.11.251.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.251.252@KRBTEST.COM; defaulting to no policy
I20260927 18:43:32.309238 2175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.301347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:39819 (local address 127.0.0.1:44182)
0927 18:43:32.301432 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.301446 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:32.301498 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:32.301798 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:32.301799 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:32.301942 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:32.301993 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:32.301994 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.302942 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.302943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:32.303448 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:32.303449 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.303958 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.303959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:32.304021 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.304558 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:32.304563 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:32.306040 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:32.308046 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:32.308048 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:32.308048 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:32.308219 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:32.308447 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:32.308447 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:32.308447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:32.308556 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:32.308915 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:32.308916 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:32.308955 (+ 39us) client_negotiation.cc:769] Sending connection context
0927 18:43:32.309037 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 18:43:32.309128 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260927 18:43:32.309376 2177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.301487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44182 (local address 127.11.251.253:39819)
0927 18:43:32.301655 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.301659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.301677 (+ 18us) server_negotiation.cc:407] Connection header received
0927 18:43:32.301703 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.301705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.301715 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.301798 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.302137 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.302842 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.303568 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.303870 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.303959 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.306149 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:32.306165 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:32.306165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:32.306176 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:32.307946 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.308303 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.308307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.308307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.308370 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.308642 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.308643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.308644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.308842 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:32.308939 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:32.309111 (+ 172us) server_negotiation.cc:299] Negotiation successful
0927 18:43:32.309183 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":113,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.252@KRBTEST.COM" created.
I20260927 18:43:32.310196 2173 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:43:32.310544 2175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.310431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:43735 (local address 127.0.0.1:42630)
0927 18:43:32.310477 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.310519 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:43735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260927 18:43:32.310670 2122 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.251.252:43735: connect: Connection refused (error 111)
I20260927 18:43:32.311137 2175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.311011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:43735 (local address 127.0.0.1:42644)
0927 18:43:32.311098 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.311120 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:43735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260927 18:43:32.311236 2173 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.252:43735: Network error: Client connection negotiation failed: client connection to 127.11.251.252:43735: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.252.keytab.
Entry for principal kudu/127.11.251.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.252.keytab.
I20260927 18:43:32.326646 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.252:43735
--webserver_interface=127.11.251.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.252.keytab
--principal=kudu/127.11.251.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:35059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:37689,127.11.251.253:39819,127.11.251.252:43735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:43:32.328781 2098 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } attempt: 3
Sep 27 18:43:32 dist-test-slave-z47j krb5kdc[2017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534612, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for kudu/127.11.251.253@KRBTEST.COM
I20260927 18:43:32.337766 2177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.329370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44196 (local address 127.11.251.253:39819)
0927 18:43:32.329457 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.329460 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.329475 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:43:32.329510 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.329512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.329530 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.329622 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.329991 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.330818 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.331632 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.332068 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.332151 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.334617 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:32.334618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:32.334619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:32.334627 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:32.336314 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.336808 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.336809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.336809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.336867 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.337118 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.337119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.337119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.337251 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:32.337311 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:32.337562 (+ 251us) server_negotiation.cc:299] Negotiation successful
0927 18:43:32.337699 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 18:43:32.337862 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.329270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:39819 (local address 127.0.0.1:44196)
0927 18:43:32.329340 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.329356 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:32.329438 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:32.329644 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:32.329645 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:32.329836 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:32.329903 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:32.329906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.330938 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.330939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:32.331525 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:32.331542 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.332165 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.332166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:32.332236 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.332693 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:32.332699 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:32.334499 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:32.336416 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:32.336417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:32.336418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:32.336699 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:32.336953 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:32.336954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:32.336955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:32.337033 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:32.337337 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:32.337339 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:32.337408 (+ 69us) client_negotiation.cc:769] Sending connection context
0927 18:43:32.337536 (+ 128us) client_negotiation.cc:240] Negotiation successful
0927 18:43:32.337685 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260927 18:43:32.338423 2098 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:43:32.338866 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.338712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:43735 (local address 127.0.0.1:42646)
0927 18:43:32.338779 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.338823 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:43735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260927 18:43:32.339277 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.339155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:43735 (local address 127.0.0.1:42652)
0927 18:43:32.339214 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.339247 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:43735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260927 18:43:32.339396 2098 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.252:43735: Network error: Client connection negotiation failed: client connection to 127.11.251.252:43735: connect: Connection refused (error 111)
I20260927 18:43:32.360459 2173 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } attempt: 1
I20260927 18:43:32.360961 2175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.360831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:43735 (local address 127.0.0.1:42658)
0927 18:43:32.360898 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.360934 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:43735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260927 18:43:32.361294 2175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.361206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:43735 (local address 127.0.0.1:42664)
0927 18:43:32.361247 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.361272 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:43735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260927 18:43:32.361389 2173 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.252:43735: Network error: Client connection negotiation failed: client connection to 127.11.251.252:43735: connect: Connection refused (error 111)
I20260927 18:43:32.370576 2098 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } attempt: 1
I20260927 18:43:32.371271 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.371055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:43735 (local address 127.0.0.1:42674)
0927 18:43:32.371176 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.371234 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:43735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260927 18:43:32.371743 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.371640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:43735 (local address 127.0.0.1:42690)
0927 18:43:32.371691 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.371727 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:43735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260927 18:43:32.371874 2098 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.252:43735: Network error: Client connection negotiation failed: client connection to 127.11.251.252:43735: connect: Connection refused (error 111)
W20260927 18:43:32.421799 2180 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 18:43:32.422066 2180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:32.422084 2180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:32.422106 2180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:32.423277 2180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:32.423316 2180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:32.423331 2180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:32.423343 2180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:32.423355 2180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:32.424844 2180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:35059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:37689,127.11.251.253:39819,127.11.251.252:43735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.252.keytab
--principal=kudu/127.11.251.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.252:43735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.251.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:32.425077 2180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:32.425280 2180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:43:32.429616 2173 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } attempt: 2
I20260927 18:43:32.430125 2175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.429977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:43735 (local address 127.0.0.1:42700)
0927 18:43:32.430048 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.430086 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:43735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260927 18:43:32.430672 2175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.430483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:43735 (local address 127.0.0.1:42708)
0927 18:43:32.430572 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.430627 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:43735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260927 18:43:32.430706 2186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:32.430740 2185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:32.430869 2173 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.252:43735: Network error: Client connection negotiation failed: client connection to 127.11.251.252:43735: connect: Connection refused (error 111)
I20260927 18:43:32.431022 2180 server_base.cc:1034] running on GCE node
W20260927 18:43:32.431151 2188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:32.431516 2180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:32.431838 2180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:32.433013 2180 hybrid_clock.cc:648] HybridClock initialized: now 1790534612432990 us; error 42 us; skew 500 ppm
Sep 27 18:43:32 dist-test-slave-z47j krb5kdc[2017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534612, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:32.435735 2180 init.cc:377] Logged in from keytab as kudu/127.11.251.252@KRBTEST.COM (short username kudu)
I20260927 18:43:32.436231 2180 webserver.cc:460] Webserver started at http://127.11.251.252:33141/ using document root <none> and password file <none>
I20260927 18:43:32.436470 2180 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:32.436534 2180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:32.436694 2180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:32.437965 2180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-2/data/instance:
uuid: "15b8f3ba40f34ce88e0b1deb44a77879"
format_stamp: "Formatted at 2026-09-27 18:43:32 on dist-test-slave-z47j"
server_key: "20741a6d18059ca55d1bfc8e0101406f"
server_key_iv: "d6c47110431a1fde78c3e0f21ab26102"
server_key_version: "encryptionkey@0"
I20260927 18:43:32.438334 2180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-2/wal/instance:
uuid: "15b8f3ba40f34ce88e0b1deb44a77879"
format_stamp: "Formatted at 2026-09-27 18:43:32 on dist-test-slave-z47j"
server_key: "20741a6d18059ca55d1bfc8e0101406f"
server_key_iv: "d6c47110431a1fde78c3e0f21ab26102"
server_key_version: "encryptionkey@0"
I20260927 18:43:32.439775 2180 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:43:32.440621 2195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:32.440864 2180 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:32.440972 2180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-2/wal
uuid: "15b8f3ba40f34ce88e0b1deb44a77879"
format_stamp: "Formatted at 2026-09-27 18:43:32 on dist-test-slave-z47j"
server_key: "20741a6d18059ca55d1bfc8e0101406f"
server_key_iv: "d6c47110431a1fde78c3e0f21ab26102"
server_key_version: "encryptionkey@0"
I20260927 18:43:32.441052 2180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:32.447060 2098 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } attempt: 2
I20260927 18:43:32.447592 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.447449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:43735 (local address 127.0.0.1:42716)
0927 18:43:32.447511 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.447560 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:43735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260927 18:43:32.447943 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.447850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:43735 (local address 127.0.0.1:42718)
0927 18:43:32.447892 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.447917 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:43735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260927 18:43:32.448024 2098 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.252:43735: Network error: Client connection negotiation failed: client connection to 127.11.251.252:43735: connect: Connection refused (error 111)
I20260927 18:43:32.453097 2180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:32.455282 2180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:32.455435 2180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:32.459859 2180 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.252:43735
I20260927 18:43:32.459913 2247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.252:43735 every 8 connection(s)
I20260927 18:43:32.460290 2180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-2/data/info.pb
I20260927 18:43:32.461310 2248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:32.463193 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 2180
I20260927 18:43:32.463310 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534329098323-12271-0/minicluster-data/master-2/wal/instance
I20260927 18:43:32.463178 2248 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37689 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:43:32.463549 12271 external_mini_cluster.cc:1428] Setting key 0a5e3047322fb68f7731d6a42b2b6a45
Sep 27 18:43:32 dist-test-slave-z47j krb5kdc[2017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534612, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.252@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
Sep 27 18:43:32 dist-test-slave-z47j krb5kdc[2017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.252@KRBTEST.COM
I20260927 18:43:32.473731 2101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.464086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56330 (local address 127.11.251.254:37689)
0927 18:43:32.464139 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.464141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.464458 (+ 317us) server_negotiation.cc:407] Connection header received
0927 18:43:32.465090 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.465091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.465102 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.465168 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.465908 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.466402 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.467435 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.467736 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.467795 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.470462 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:32.470464 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:32.470464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:32.470470 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:32.472131 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.472694 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.472695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.472695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.472755 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.473009 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.473010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.473010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.473130 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:32.473202 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:32.473490 (+ 288us) server_negotiation.cc:299] Negotiation successful
0927 18:43:32.473570 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 18:43:32.474213 2252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.464096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:37689 (local address 127.0.0.1:56330)
0927 18:43:32.464320 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.464364 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:32.464946 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:32.465211 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:32.465213 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:32.465525 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:32.465795 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:32.465800 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.466486 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.466487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:32.467351 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:32.467353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.467813 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.467814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:32.467954 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.468634 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:32.468643 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:32.470349 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:32.472236 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:32.472247 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:32.472249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:32.472614 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:32.472821 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:32.472822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:32.472822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:32.472942 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:32.473196 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:32.473203 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:32.473359 (+ 156us) client_negotiation.cc:769] Sending connection context
0927 18:43:32.473470 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 18:43:32.473714 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":129,"threads_started":1}
I20260927 18:43:32.474216 2251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.464097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42720 (local address 127.11.251.252:43735)
0927 18:43:32.464311 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.464323 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.464363 (+ 40us) server_negotiation.cc:407] Connection header received
0927 18:43:32.464942 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.464952 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.464982 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.465100 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.465790 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.466824 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.467681 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.468087 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.468165 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.471020 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:32.471039 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:32.471040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:32.471047 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:32.472763 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.473161 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.473165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.473165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.473229 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.473435 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.473436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.473436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.473651 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:32.473821 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:32.473937 (+ 116us) server_negotiation.cc:299] Negotiation successful
0927 18:43:32.474025 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":104,"threads_started":1}
I20260927 18:43:32.474963 2248 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:43:32 dist-test-slave-z47j krb5kdc[2017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534612, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.252@KRBTEST.COM for kudu/127.11.251.253@KRBTEST.COM
Sep 27 18:43:32 dist-test-slave-z47j krb5kdc[2017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:43:32.482739 2252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.475278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:39819 (local address 127.0.0.1:44200)
0927 18:43:32.475337 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.475350 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:32.475397 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:32.475565 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:32.475566 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:32.475743 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:32.475804 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:32.475806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.476453 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.476454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:32.476980 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:32.476981 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.477370 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.477371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:32.477439 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.477884 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:32.477889 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:32.479314 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:32.481377 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:32.481378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:32.481378 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:32.481626 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:32.481848 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:32.481849 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:32.481849 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:32.481916 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:32.482372 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:32.482373 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:32.482411 (+ 38us) client_negotiation.cc:769] Sending connection context
0927 18:43:32.482498 (+ 87us) client_negotiation.cc:240] Negotiation successful
0927 18:43:32.482611 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260927 18:43:32.482746 2177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.475382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44200 (local address 127.11.251.253:39819)
0927 18:43:32.475443 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.475445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.475452 (+ 7us) server_negotiation.cc:407] Connection header received
0927 18:43:32.475476 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.475477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.475487 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.475550 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.475873 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.476373 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.477072 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.477290 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.477372 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.479427 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:32.479430 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:32.479430 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:32.479439 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:32.481277 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.481718 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.481719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.481720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.481773 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.482068 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.482069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.482070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.482272 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:32.482365 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:32.482527 (+ 162us) server_negotiation.cc:299] Negotiation successful
0927 18:43:32.482623 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 18:43:32.483392 2101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.475179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56344 (local address 127.11.251.254:37689)
0927 18:43:32.475256 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.475259 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.475395 (+ 136us) server_negotiation.cc:407] Connection header received
0927 18:43:32.475608 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.475609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.475619 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.475703 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.476219 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.476718 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.477407 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.477693 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.477770 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.480344 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:32.480345 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:32.480345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:32.480352 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:32.482066 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.482472 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.482473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.482473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.482531 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.482766 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.482767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.482767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.482904 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:32.483043 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:32.483197 (+ 154us) server_negotiation.cc:299] Negotiation successful
0927 18:43:32.483290 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:43:32.483486 2248 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:43:32 dist-test-slave-z47j krb5kdc[2017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534612, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.252@KRBTEST.COM for kudu/127.11.251.252@KRBTEST.COM
Sep 27 18:43:32 dist-test-slave-z47j krb5kdc[2017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.253@KRBTEST.COM
I20260927 18:43:32.491529 2252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.483802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:43735 (local address 127.0.0.1:42724)
0927 18:43:32.483847 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.483856 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:32.483893 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:32.484090 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:32.484091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:32.484268 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:32.484313 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:32.484315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.485282 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.485283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:32.485995 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:32.485997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.486529 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.486530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:32.486590 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.487066 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:32.487071 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:32.488574 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:32.490345 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:32.490347 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:32.490348 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:32.490567 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:32.490804 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:32.490804 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:32.490805 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:32.490871 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:32.491159 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:32.491160 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:32.491205 (+ 45us) client_negotiation.cc:769] Sending connection context
0927 18:43:32.491288 (+ 83us) client_negotiation.cc:240] Negotiation successful
0927 18:43:32.491413 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260927 18:43:32.491544 2251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.483855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42724 (local address 127.11.251.252:43735)
0927 18:43:32.483895 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.483897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.483962 (+ 65us) server_negotiation.cc:407] Connection header received
0927 18:43:32.483987 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.483987 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.483998 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.484049 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.484399 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.485104 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.486071 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.486397 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.486479 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.488668 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:32.488669 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:32.488670 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:32.488677 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:32.490257 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.490665 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.490666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.490666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.490720 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.490950 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.490951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.490951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.491071 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:32.491167 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:32.491322 (+ 155us) server_negotiation.cc:299] Negotiation successful
0927 18:43:32.491462 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":78,"server-negotiator.queue_time_us":17}
I20260927 18:43:32.493394 2248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15b8f3ba40f34ce88e0b1deb44a77879: Bootstrap starting.
I20260927 18:43:32.494100 2177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.484078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44208 (local address 127.11.251.253:39819)
0927 18:43:32.484129 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.484131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.484140 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:43:32.484206 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.484207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.484217 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.484283 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.484913 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.485675 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.486527 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.486816 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.486869 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.489292 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:32.489293 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:32.489294 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:32.489301 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:32.491058 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.491380 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.491381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.491382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.491434 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.492096 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.492097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.492098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.492200 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:32.492389 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:32.493888 (+ 1499us) server_negotiation.cc:299] Negotiation successful
0927 18:43:32.493977 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 18:43:32.494644 2248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15b8f3ba40f34ce88e0b1deb44a77879: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:32.494962 2248 log.cc:824] T 00000000000000000000000000000000 P 15b8f3ba40f34ce88e0b1deb44a77879: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:32.496060 2248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15b8f3ba40f34ce88e0b1deb44a77879: No bootstrap required, opened a new log
I20260927 18:43:32.497112 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:43:32.497995 2248 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15b8f3ba40f34ce88e0b1deb44a77879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2934c5e4bad94a80b2becc163faebec3" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37689 } } peers { permanent_uuid: "33b7bca518414fb6991f864f4b20ac1c" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } } peers { permanent_uuid: "15b8f3ba40f34ce88e0b1deb44a77879" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } }
I20260927 18:43:32.498139 2248 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15b8f3ba40f34ce88e0b1deb44a77879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15b8f3ba40f34ce88e0b1deb44a77879, State: Initialized, Role: FOLLOWER
I20260927 18:43:32.498225 2248 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15b8f3ba40f34ce88e0b1deb44a77879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2934c5e4bad94a80b2becc163faebec3" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37689 } } peers { permanent_uuid: "33b7bca518414fb6991f864f4b20ac1c" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } } peers { permanent_uuid: "15b8f3ba40f34ce88e0b1deb44a77879" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } }
I20260927 18:43:32.498548 2254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15b8f3ba40f34ce88e0b1deb44a77879 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2934c5e4bad94a80b2becc163faebec3" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37689 } } peers { permanent_uuid: "33b7bca518414fb6991f864f4b20ac1c" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } } peers { permanent_uuid: "15b8f3ba40f34ce88e0b1deb44a77879" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } } }
I20260927 18:43:32.498620 2248 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15b8f3ba40f34ce88e0b1deb44a77879 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:32.498672 2254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15b8f3ba40f34ce88e0b1deb44a77879 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:43:32.498334 2072 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2934c5e4bad94a80b2becc163faebec3"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56344
W20260927 18:43:32.500660 2265 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 15b8f3ba40f34ce88e0b1deb44a77879: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:43:32.500738 2265 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:43:32.501029 2180 master_runner.cc:186] Error getting master registration for 127.11.251.254:37689: OK, instance_id { permanent_uuid: "2934c5e4bad94a80b2becc163faebec3" instance_seqno: 1790534612098846 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 18:43:32.503374 2180 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 18:43:32.522032 2173 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } attempt: 3
Sep 27 18:43:32 dist-test-slave-z47j krb5kdc[2017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534612, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.253@KRBTEST.COM for kudu/127.11.251.252@KRBTEST.COM
I20260927 18:43:32.531183 2251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.522737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42740 (local address 127.11.251.252:43735)
0927 18:43:32.522806 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.522809 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.522821 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:43:32.522854 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.522855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.522867 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.522966 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.523396 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.524184 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.525441 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.525788 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.525870 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.528065 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:32.528067 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:32.528067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:32.528074 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:32.529807 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.530265 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.530265 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.530266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.530322 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.530537 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.530538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.530538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.530726 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:32.530798 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:32.530986 (+ 188us) server_negotiation.cc:299] Negotiation successful
0927 18:43:32.531080 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 18:43:32.531208 2175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.522508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:43735 (local address 127.0.0.1:42740)
0927 18:43:32.522582 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.522595 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:32.522643 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:32.522997 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:32.522998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:32.523227 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:32.523308 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:32.523311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.524301 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.524303 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:32.525350 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:32.525353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.525868 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.525869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:32.525927 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.526419 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:32.526424 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:32.527950 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:32.529904 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:32.529905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:32.529906 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:32.530177 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:32.530395 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:32.530396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:32.530397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:32.530466 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:32.530800 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:32.530802 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:32.530867 (+ 65us) client_negotiation.cc:769] Sending connection context
0927 18:43:32.530953 (+ 86us) client_negotiation.cc:240] Negotiation successful
0927 18:43:32.531087 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260927 18:43:32.533530 2173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33b7bca518414fb6991f864f4b20ac1c: Bootstrap starting.
I20260927 18:43:32.534425 2173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33b7bca518414fb6991f864f4b20ac1c: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:32.534761 2173 log.cc:824] T 00000000000000000000000000000000 P 33b7bca518414fb6991f864f4b20ac1c: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:32.535816 2173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33b7bca518414fb6991f864f4b20ac1c: No bootstrap required, opened a new log
I20260927 18:43:32.538147 2173 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33b7bca518414fb6991f864f4b20ac1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2934c5e4bad94a80b2becc163faebec3" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37689 } } peers { permanent_uuid: "33b7bca518414fb6991f864f4b20ac1c" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } } peers { permanent_uuid: "15b8f3ba40f34ce88e0b1deb44a77879" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } }
I20260927 18:43:32.538314 2173 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33b7bca518414fb6991f864f4b20ac1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b7bca518414fb6991f864f4b20ac1c, State: Initialized, Role: FOLLOWER
I20260927 18:43:32.538407 2173 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33b7bca518414fb6991f864f4b20ac1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2934c5e4bad94a80b2becc163faebec3" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37689 } } peers { permanent_uuid: "33b7bca518414fb6991f864f4b20ac1c" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } } peers { permanent_uuid: "15b8f3ba40f34ce88e0b1deb44a77879" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } }
I20260927 18:43:32.538789 2173 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33b7bca518414fb6991f864f4b20ac1c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:32.538759 2268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33b7bca518414fb6991f864f4b20ac1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2934c5e4bad94a80b2becc163faebec3" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37689 } } peers { permanent_uuid: "33b7bca518414fb6991f864f4b20ac1c" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } } peers { permanent_uuid: "15b8f3ba40f34ce88e0b1deb44a77879" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } } }
I20260927 18:43:32.538913 2268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33b7bca518414fb6991f864f4b20ac1c [sys.catalog]: This master's current role is: FOLLOWER
W20260927 18:43:32.540993 2279 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 33b7bca518414fb6991f864f4b20ac1c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:43:32.541074 2279 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:43:32.541350 2105 master_runner.cc:186] Error getting master registration for 127.11.251.254:37689: OK, instance_id { permanent_uuid: "2934c5e4bad94a80b2becc163faebec3" instance_seqno: 1790534612098846 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 18:43:32.544029 2105 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 18:43:32.547176 2098 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } attempt: 3
Sep 27 18:43:32 dist-test-slave-z47j krb5kdc[2017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534612, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for kudu/127.11.251.252@KRBTEST.COM
I20260927 18:43:32.556332 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.547520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:43735 (local address 127.0.0.1:42750)
0927 18:43:32.547583 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:32.547595 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:32.547638 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:32.547989 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:32.547990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:32.548196 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:32.548255 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:32.548257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.549255 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.549256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:32.550123 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:32.550126 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.550605 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.550607 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:32.550669 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.551255 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:32.551261 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:32.552913 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:32.555097 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:32.555098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:32.555099 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:32.555359 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:32.555594 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:32.555595 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:32.555595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:32.555663 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:32.555966 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:32.555967 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:32.556018 (+ 51us) client_negotiation.cc:769] Sending connection context
0927 18:43:32.556121 (+ 103us) client_negotiation.cc:240] Negotiation successful
0927 18:43:32.556221 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260927 18:43:32.556372 2251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.547720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42750 (local address 127.11.251.252:43735)
0927 18:43:32.547791 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.547795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.547806 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:43:32.547837 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.547839 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.547851 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.547963 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.548341 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.549158 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.550208 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.550516 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.550594 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.553082 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:32.553084 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:32.553085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:32.553093 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:32.554927 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.555441 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.555443 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.555443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.555519 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.555734 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.555735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.555736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.555890 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:32.555966 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:32.556171 (+ 205us) server_negotiation.cc:299] Negotiation successful
0927 18:43:32.556266 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 18:43:32.558444 2098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3: Bootstrap starting.
I20260927 18:43:32.559221 2098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:32.559521 2098 log.cc:824] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:32.560670 2098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3: No bootstrap required, opened a new log
I20260927 18:43:32.562121 2098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2934c5e4bad94a80b2becc163faebec3" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37689 } } peers { permanent_uuid: "33b7bca518414fb6991f864f4b20ac1c" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } } peers { permanent_uuid: "15b8f3ba40f34ce88e0b1deb44a77879" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } }
I20260927 18:43:32.562278 2098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2934c5e4bad94a80b2becc163faebec3, State: Initialized, Role: FOLLOWER
I20260927 18:43:32.562364 2098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2934c5e4bad94a80b2becc163faebec3" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37689 } } peers { permanent_uuid: "33b7bca518414fb6991f864f4b20ac1c" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } } peers { permanent_uuid: "15b8f3ba40f34ce88e0b1deb44a77879" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } }
I20260927 18:43:32.562673 2098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:32.563009 2072 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260927 18:43:32.563112 2072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:32.564136 2072 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2934c5e4bad94a80b2becc163faebec3" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37689 } } peers { permanent_uuid: "33b7bca518414fb6991f864f4b20ac1c" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } } peers { permanent_uuid: "15b8f3ba40f34ce88e0b1deb44a77879" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } }
I20260927 18:43:32.564484 2072 leader_election.cc:288] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [CANDIDATE]: Term 1 election: Requested vote from peers 33b7bca518414fb6991f864f4b20ac1c (127.11.251.253:39819), 15b8f3ba40f34ce88e0b1deb44a77879 (127.11.251.252:43735)
I20260927 18:43:32.565136 2147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2934c5e4bad94a80b2becc163faebec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "33b7bca518414fb6991f864f4b20ac1c"
I20260927 18:43:32.565280 2147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33b7bca518414fb6991f864f4b20ac1c [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:32.566385 2147 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 33b7bca518414fb6991f864f4b20ac1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2934c5e4bad94a80b2becc163faebec3 in term 1.
W20260927 18:43:32.566474 2293 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:43:32.566529 2293 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:43:32.566627 2222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2934c5e4bad94a80b2becc163faebec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "15b8f3ba40f34ce88e0b1deb44a77879"
I20260927 18:43:32.566749 2222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15b8f3ba40f34ce88e0b1deb44a77879 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:32.567648 2222 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 15b8f3ba40f34ce88e0b1deb44a77879 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2934c5e4bad94a80b2becc163faebec3 in term 1.
I20260927 18:43:32.568573 2047 leader_election.cc:302] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15b8f3ba40f34ce88e0b1deb44a77879, 2934c5e4bad94a80b2becc163faebec3; no voters:
I20260927 18:43:32.570591 2282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2934c5e4bad94a80b2becc163faebec3" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37689 } } peers { permanent_uuid: "33b7bca518414fb6991f864f4b20ac1c" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } } peers { permanent_uuid: "15b8f3ba40f34ce88e0b1deb44a77879" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } } }
I20260927 18:43:32.570807 2282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:43:32.570920 2282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:32.571061 2282 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [term 1 LEADER]: Becoming Leader. State: Replica: 2934c5e4bad94a80b2becc163faebec3, State: Running, Role: LEADER
I20260927 18:43:32.571280 2030 master_runner.cc:263] No leader master found from master 2934c5e4bad94a80b2becc163faebec3
I20260927 18:43:32.571326 2030 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 18:43:32.571249 2282 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2934c5e4bad94a80b2becc163faebec3" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37689 } } peers { permanent_uuid: "33b7bca518414fb6991f864f4b20ac1c" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } } peers { permanent_uuid: "15b8f3ba40f34ce88e0b1deb44a77879" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } }
I20260927 18:43:32.571904 2282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2934c5e4bad94a80b2becc163faebec3. Latest consensus state: current_term: 1 leader_uuid: "2934c5e4bad94a80b2becc163faebec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2934c5e4bad94a80b2becc163faebec3" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37689 } } peers { permanent_uuid: "33b7bca518414fb6991f864f4b20ac1c" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } } peers { permanent_uuid: "15b8f3ba40f34ce88e0b1deb44a77879" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } } }
I20260927 18:43:32.572052 2282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:32.572316 2301 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:32.573046 2301 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:32.576020 2222 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 15b8f3ba40f34ce88e0b1deb44a77879 [term 1 FOLLOWER]: Refusing update from remote peer 2934c5e4bad94a80b2becc163faebec3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:32.576050 2147 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 33b7bca518414fb6991f864f4b20ac1c [term 1 FOLLOWER]: Refusing update from remote peer 2934c5e4bad94a80b2becc163faebec3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:32.576336 2282 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15b8f3ba40f34ce88e0b1deb44a77879" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:32.576530 2299 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b7bca518414fb6991f864f4b20ac1c" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:32.577754 2254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15b8f3ba40f34ce88e0b1deb44a77879 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2934c5e4bad94a80b2becc163faebec3. Latest consensus state: current_term: 1 leader_uuid: "2934c5e4bad94a80b2becc163faebec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2934c5e4bad94a80b2becc163faebec3" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37689 } } peers { permanent_uuid: "33b7bca518414fb6991f864f4b20ac1c" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } } peers { permanent_uuid: "15b8f3ba40f34ce88e0b1deb44a77879" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } } }
I20260927 18:43:32.577864 2254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15b8f3ba40f34ce88e0b1deb44a77879 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:43:32.578217 2268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33b7bca518414fb6991f864f4b20ac1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2934c5e4bad94a80b2becc163faebec3. Latest consensus state: current_term: 1 leader_uuid: "2934c5e4bad94a80b2becc163faebec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2934c5e4bad94a80b2becc163faebec3" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37689 } } peers { permanent_uuid: "33b7bca518414fb6991f864f4b20ac1c" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } } peers { permanent_uuid: "15b8f3ba40f34ce88e0b1deb44a77879" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } } }
I20260927 18:43:32.578310 2268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33b7bca518414fb6991f864f4b20ac1c [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:43:32.579425 2282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2934c5e4bad94a80b2becc163faebec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2934c5e4bad94a80b2becc163faebec3" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37689 } } peers { permanent_uuid: "33b7bca518414fb6991f864f4b20ac1c" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } } peers { permanent_uuid: "15b8f3ba40f34ce88e0b1deb44a77879" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } } }
I20260927 18:43:32.579519 2282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:32.579500 2268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33b7bca518414fb6991f864f4b20ac1c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2934c5e4bad94a80b2becc163faebec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2934c5e4bad94a80b2becc163faebec3" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37689 } } peers { permanent_uuid: "33b7bca518414fb6991f864f4b20ac1c" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } } peers { permanent_uuid: "15b8f3ba40f34ce88e0b1deb44a77879" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } } }
I20260927 18:43:32.579574 2268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33b7bca518414fb6991f864f4b20ac1c [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:43:32.579658 2302 mvcc.cc:205] Tried to move back new op lower bound from 7334029773108637696 to 7334029773093232640. Current Snapshot: MvccSnapshot[applied={T|T < 7334029773108637696}]
I20260927 18:43:32.580318 2308 mvcc.cc:205] Tried to move back new op lower bound from 7334029773108637696 to 7334029773093232640. Current Snapshot: MvccSnapshot[applied={T|T < 7334029773108637696}]
I20260927 18:43:32.582962 2306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2934c5e4bad94a80b2becc163faebec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2934c5e4bad94a80b2becc163faebec3" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37689 } } peers { permanent_uuid: "33b7bca518414fb6991f864f4b20ac1c" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } } peers { permanent_uuid: "15b8f3ba40f34ce88e0b1deb44a77879" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } } }
I20260927 18:43:32.583074 2306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:32.583112 2301 catalog_manager.cc:1347] Generated new cluster ID: 9307152f204041d383dd9e4b858826c3
I20260927 18:43:32.583146 2301 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:32.583510 2254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15b8f3ba40f34ce88e0b1deb44a77879 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2934c5e4bad94a80b2becc163faebec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2934c5e4bad94a80b2becc163faebec3" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 37689 } } peers { permanent_uuid: "33b7bca518414fb6991f864f4b20ac1c" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39819 } } peers { permanent_uuid: "15b8f3ba40f34ce88e0b1deb44a77879" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 43735 } } }
I20260927 18:43:32.583585 2254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15b8f3ba40f34ce88e0b1deb44a77879 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:43:32.583809 2307 mvcc.cc:205] Tried to move back new op lower bound from 7334029773108637696 to 7334029773093232640. Current Snapshot: MvccSnapshot[applied={T|T < 7334029773108637696}]
I20260927 18:43:32.619385 2101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.574670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56352 (local address 127.11.251.254:37689)
0927 18:43:32.574711 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.574713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.583022 (+ 8309us) server_negotiation.cc:407] Connection header received
0927 18:43:32.593916 (+ 10894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.593919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.593934 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.593981 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.595086 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.595801 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.604934 (+ 9133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.605278 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.605336 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.613075 (+ 7739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:32.613078 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:32.613078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:32.613089 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:32.615026 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.615495 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.615497 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.615498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.615563 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.615783 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.615784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.615784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.615944 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:32.616059 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:32.619168 (+ 3109us) server_negotiation.cc:299] Negotiation successful
0927 18:43:32.619267 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 18:43:32.623100 2251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.574457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42754 (local address 127.11.251.252:43735)
0927 18:43:32.574495 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.574497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.595032 (+ 20535us) server_negotiation.cc:407] Connection header received
0927 18:43:32.596965 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.596967 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.596981 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.597016 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.602194 (+ 5178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.602874 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.608983 (+ 6109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.609315 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.609381 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.617193 (+ 7812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:32.617196 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:32.617196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:32.617206 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:32.619009 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.622105 (+ 3096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.622106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.622106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.622175 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.622396 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.622397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.622397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.622564 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:32.622658 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:32.622893 (+ 235us) server_negotiation.cc:299] Negotiation successful
0927 18:43:32.622985 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 18:43:32.626572 2301 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:32.627547 2301 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:32.628700 2177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:32.574241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44218 (local address 127.11.251.253:39819)
0927 18:43:32.574291 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.574294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.608063 (+ 33769us) server_negotiation.cc:407] Connection header received
0927 18:43:32.608186 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.608187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.608203 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.608238 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.611257 (+ 3019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.612020 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.616281 (+ 4261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:32.616612 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:32.616666 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:32.624281 (+ 7615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:32.624283 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:32.624284 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:32.624295 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:32.626131 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.627728 (+ 1597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.627729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.627730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.627800 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:32.628044 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:32.628045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:32.628046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:32.628194 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:32.628290 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:32.628509 (+ 219us) server_negotiation.cc:299] Negotiation successful
0927 18:43:32.628601 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:43:32.643877 2301 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2934c5e4bad94a80b2becc163faebec3: Generated new TSK 0
I20260927 18:43:32.644271 2301 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 18:43:32.672307 2101 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:32.670675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56364 (local address 127.11.251.254:37689)
0927 18:43:32.670732 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.670735 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.671499 (+ 764us) server_negotiation.cc:407] Connection header received
0927 18:43:32.671625 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.671626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.671639 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.671674 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.672217 (+ 543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56364: BlockingRecv error: recv got EOF from 127.0.0.1:56364 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260927 18:43:32.673357 2251 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:32.671177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42766 (local address 127.11.251.252:43735)
0927 18:43:32.671223 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.671225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.672807 (+ 1582us) server_negotiation.cc:407] Connection header received
0927 18:43:32.672909 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.672910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.672934 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.672966 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.673289 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42766: BlockingRecv error: recv got EOF from 127.0.0.1:42766 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260927 18:43:32.674590 2177 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:32.673902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44224 (local address 127.11.251.253:39819)
0927 18:43:32.673960 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.673962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.674079 (+ 117us) server_negotiation.cc:407] Connection header received
0927 18:43:32.674175 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.674176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.674187 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.674216 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.674524 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44224: BlockingRecv error: recv got EOF from 127.0.0.1:44224 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260927 18:43:32.677254 2177 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:32.676507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44232 (local address 127.11.251.253:39819)
0927 18:43:32.676545 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.676546 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.676722 (+ 176us) server_negotiation.cc:407] Connection header received
0927 18:43:32.676829 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.676830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.676851 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.676879 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.677211 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44232: BlockingRecv error: recv got EOF from 127.0.0.1:44232 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260927 18:43:32.680446 2177 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:32.679758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44242 (local address 127.11.251.253:39819)
0927 18:43:32.679790 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.679791 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.680005 (+ 214us) server_negotiation.cc:407] Connection header received
0927 18:43:32.680089 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.680090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.680099 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.680127 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.680405 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44242: BlockingRecv error: recv got EOF from 127.0.0.1:44242 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260927 18:43:32.686419 2177 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:32.685013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44246 (local address 127.11.251.253:39819)
0927 18:43:32.685053 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.685055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.685900 (+ 845us) server_negotiation.cc:407] Connection header received
0927 18:43:32.686010 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.686011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.686021 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.686051 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.686369 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44246: BlockingRecv error: recv got EOF from 127.0.0.1:44246 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260927 18:43:32.692862 2177 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:32.691896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44250 (local address 127.11.251.253:39819)
0927 18:43:32.691957 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.691959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.692308 (+ 349us) server_negotiation.cc:407] Connection header received
0927 18:43:32.692408 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.692409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.692420 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.692454 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.692807 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44250: BlockingRecv error: recv got EOF from 127.0.0.1:44250 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260927 18:43:32.703212 2177 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:32.702433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44264 (local address 127.11.251.253:39819)
0927 18:43:32.702489 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.702491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.702633 (+ 142us) server_negotiation.cc:407] Connection header received
0927 18:43:32.702709 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.702710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.702723 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.702789 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.703135 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44264: BlockingRecv error: recv got EOF from 127.0.0.1:44264 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260927 18:43:32.721305 2177 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:32.720454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44270 (local address 127.11.251.253:39819)
0927 18:43:32.720511 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.720514 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.720681 (+ 167us) server_negotiation.cc:407] Connection header received
0927 18:43:32.720773 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.720774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.720791 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.720850 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.721220 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44270: BlockingRecv error: recv got EOF from 127.0.0.1:44270 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260927 18:43:32.755693 2177 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:32.754775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44274 (local address 127.11.251.253:39819)
0927 18:43:32.754879 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.754881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.755015 (+ 134us) server_negotiation.cc:407] Connection header received
0927 18:43:32.755111 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.755113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.755127 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.755182 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.755617 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44274: BlockingRecv error: recv got EOF from 127.0.0.1:44274 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260927 18:43:32.822330 2177 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:32.821224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44286 (local address 127.11.251.253:39819)
0927 18:43:32.821302 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.821304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.821485 (+ 181us) server_negotiation.cc:407] Connection header received
0927 18:43:32.821559 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.821560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.821624 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.821690 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.822235 (+ 545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44286: BlockingRecv error: recv got EOF from 127.0.0.1:44286 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260927 18:43:32.952592 2177 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:32.951665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44302 (local address 127.11.251.253:39819)
0927 18:43:32.951784 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:32.951786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:32.951838 (+ 52us) server_negotiation.cc:407] Connection header received
0927 18:43:32.951931 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:32.951932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:32.951945 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:32.952009 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:32.952485 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44302: BlockingRecv error: recv got EOF from 127.0.0.1:44302 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260927 18:43:33.210893 2177 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:33.210007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44306 (local address 127.11.251.253:39819)
0927 18:43:33.210114 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:33.210116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:33.210198 (+ 82us) server_negotiation.cc:407] Connection header received
0927 18:43:33.210236 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:33.210237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:33.210249 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:33.210316 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:33.210827 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44306: BlockingRecv error: recv got EOF from 127.0.0.1:44306 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260927 18:43:33.501258 2265 catalog_manager.cc:1259] Loaded cluster ID: 9307152f204041d383dd9e4b858826c3
I20260927 18:43:33.501379 2265 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 15b8f3ba40f34ce88e0b1deb44a77879: loading cluster ID for follower catalog manager: success
I20260927 18:43:33.502266 2265 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 15b8f3ba40f34ce88e0b1deb44a77879: acquiring CA information for follower catalog manager: success
I20260927 18:43:33.502573 2265 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 15b8f3ba40f34ce88e0b1deb44a77879: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 18:43:33.541615 2279 catalog_manager.cc:1259] Loaded cluster ID: 9307152f204041d383dd9e4b858826c3
I20260927 18:43:33.541700 2279 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 33b7bca518414fb6991f864f4b20ac1c: loading cluster ID for follower catalog manager: success
I20260927 18:43:33.542485 2279 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 33b7bca518414fb6991f864f4b20ac1c: acquiring CA information for follower catalog manager: success
I20260927 18:43:33.542758 2279 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 33b7bca518414fb6991f864f4b20ac1c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 18:43:33.726877 2377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:33.724143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44312 (local address 127.11.251.253:39819)
0927 18:43:33.724306 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:33.724309 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:33.724364 (+ 55us) server_negotiation.cc:407] Connection header received
0927 18:43:33.724473 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:33.724474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:33.724541 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:33.724622 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:33.724768 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:33.725318 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:33.726077 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:33.726269 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:33.726325 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:33.726490 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:33.726614 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:33.726681 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:33.726724 (+ 43us) server_negotiation.cc:299] Negotiation successful
0927 18:43:33.726743 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":82,"threads_started":1}
I20260927 18:43:33.731541 2383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:33.728896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42776 (local address 127.11.251.252:43735)
0927 18:43:33.729018 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:33.729020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:33.729051 (+ 31us) server_negotiation.cc:407] Connection header received
0927 18:43:33.729094 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:33.729095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:33.729138 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:33.729212 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:33.729321 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:33.729923 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:33.730681 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:33.730879 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:33.730957 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:33.731163 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:33.731283 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:33.731349 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:33.731442 (+ 93us) server_negotiation.cc:299] Negotiation successful
0927 18:43:33.731461 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":61,"threads_started":1}
I20260927 18:43:33.736399 2377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:33.733449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44326 (local address 127.11.251.253:39819)
0927 18:43:33.733597 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:33.733599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:33.733754 (+ 155us) server_negotiation.cc:407] Connection header received
0927 18:43:33.733879 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:33.733881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:33.733896 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:33.733942 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:33.734161 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:33.734690 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:33.735595 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:33.735819 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:33.735898 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:33.736094 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:33.736135 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:33.736199 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:33.736282 (+ 83us) server_negotiation.cc:299] Negotiation successful
0927 18:43:33.736314 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260927 18:43:33.736716 2383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:33.733518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42790 (local address 127.11.251.252:43735)
0927 18:43:33.733566 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:33.733568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:33.733870 (+ 302us) server_negotiation.cc:407] Connection header received
0927 18:43:33.733915 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:33.733916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:33.733933 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:33.733987 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:33.734205 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:33.734669 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:33.735886 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:33.736154 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:33.736229 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:33.736415 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:33.736452 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:33.736482 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:33.736600 (+ 118us) server_negotiation.cc:299] Negotiation successful
0927 18:43:33.736627 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 18:43:33.737880 2389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:33.733446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56370 (local address 127.11.251.254:37689)
0927 18:43:33.733595 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:33.733597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:33.733732 (+ 135us) server_negotiation.cc:407] Connection header received
0927 18:43:33.733836 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:33.733837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:33.733897 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:33.733994 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:33.734815 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:33.735571 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:33.736761 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:33.737210 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:33.737273 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:33.737498 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:33.737641 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:33.737703 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:33.737755 (+ 52us) server_negotiation.cc:299] Negotiation successful
0927 18:43:33.737777 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":66,"threads_started":1}
I20260927 18:43:33.746217 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 2030
I20260927 18:43:33.752709 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 2105
I20260927 18:43:33.758605 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 2180
2026-09-27T18:43:33Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2975 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:33 dist-test-slave-z47j krb5kdc[2394](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:33 dist-test-slave-z47j krb5kdc[2394](info): set up 2 sockets
Sep 27 18:43:33 dist-test-slave-z47j krb5kdc[2394](info): commencing operation
krb5kdc: starting...
W20260927 18:43:34.798792 12271 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/test-user.keytab.
Sep 27 18:43:34 dist-test-slave-z47j krb5kdc[2394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.251.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
Entry for principal kudu/127.11.251.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab.
I20260927 18:43:34.933176 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:43369
--webserver_interface=127.11.251.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:44695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:43369,127.11.251.253:39595,127.11.251.252:40495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:35.018342 2407 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 18:43:35.018592 2407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:35.018623 2407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:35.018664 2407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:35.019830 2407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:35.019896 2407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:35.019922 2407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:35.019953 2407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:35.019984 2407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:35.021474 2407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:44695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:43369,127.11.251.253:39595,127.11.251.252:40495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.254.keytab
--principal=kudu/127.11.251.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.254:43369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.251.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:35.021782 2407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:35.022014 2407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:35.027743 2413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:35.027961 2407 server_base.cc:1034] running on GCE node
W20260927 18:43:35.027758 2412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:35.027741 2415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:35.028404 2407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:35.028717 2407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:35.029891 2407 hybrid_clock.cc:648] HybridClock initialized: now 1790534615029877 us; error 41 us; skew 500 ppm
Sep 27 18:43:35 dist-test-slave-z47j krb5kdc[2394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534615, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:35.032481 2407 init.cc:377] Logged in from keytab as kudu/127.11.251.254@KRBTEST.COM (short username kudu)
I20260927 18:43:35.032946 2407 webserver.cc:460] Webserver started at http://127.11.251.254:40373/ using document root <none> and password file <none>
I20260927 18:43:35.033208 2407 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:35.033259 2407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:35.033414 2407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:35.034575 2407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-0/data/instance:
uuid: "0dadb983ce6e438881d47ced6728242a"
format_stamp: "Formatted at 2026-09-27 18:43:35 on dist-test-slave-z47j"
server_key: "3aed857f0165f6bc8eea89d4a713833f"
server_key_iv: "d7e1e9c3cc985afcb6b666f6ce46bb55"
server_key_version: "encryptionkey@0"
I20260927 18:43:35.034958 2407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-0/wal/instance:
uuid: "0dadb983ce6e438881d47ced6728242a"
format_stamp: "Formatted at 2026-09-27 18:43:35 on dist-test-slave-z47j"
server_key: "3aed857f0165f6bc8eea89d4a713833f"
server_key_iv: "d7e1e9c3cc985afcb6b666f6ce46bb55"
server_key_version: "encryptionkey@0"
I20260927 18:43:35.036377 2407 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:43:35.037180 2422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:35.037395 2407 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:35.037489 2407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-0/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-0/wal
uuid: "0dadb983ce6e438881d47ced6728242a"
format_stamp: "Formatted at 2026-09-27 18:43:35 on dist-test-slave-z47j"
server_key: "3aed857f0165f6bc8eea89d4a713833f"
server_key_iv: "d7e1e9c3cc985afcb6b666f6ce46bb55"
server_key_version: "encryptionkey@0"
I20260927 18:43:35.037570 2407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:35.060105 2407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:35.062412 2407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:35.062572 2407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:35.067070 2407 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.254:43369
I20260927 18:43:35.067194 2474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.254:43369 every 8 connection(s)
I20260927 18:43:35.067543 2407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:35.068785 2475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:35.069260 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 2407
I20260927 18:43:35.069386 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-0/wal/instance
I20260927 18:43:35.069700 12271 external_mini_cluster.cc:1428] Setting key 10c7af552b4fdc96a4c0a3fe8d39a915
I20260927 18:43:35.071903 2475 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.254" port: 43369 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.251.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.253@KRBTEST.COM" created.
Sep 27 18:43:35 dist-test-slave-z47j krb5kdc[2394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534615, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:43:35.086012 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.072809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:43369 (local address 127.0.0.1:47474)
0927 18:43:35.073004 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.073068 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:35.073841 (+ 773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:35.074233 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:35.074236 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:35.074662 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:35.074926 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:35.074931 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.076693 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.076695 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:35.077527 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:35.077530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.078406 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.078407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:35.078484 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.079141 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:35.079151 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:35.082555 (+ 3404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:35.084501 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:35.084508 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:35.084510 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:35.084707 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:35.084937 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:35.084938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:35.084939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:35.085019 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:35.085371 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:35.085376 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:35.085439 (+ 63us) client_negotiation.cc:769] Sending connection context
0927 18:43:35.085546 (+ 107us) client_negotiation.cc:240] Negotiation successful
0927 18:43:35.085764 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260927 18:43:35.086012 2479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.072895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47474 (local address 127.11.251.254:43369)
0927 18:43:35.073099 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.073104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.073199 (+ 95us) server_negotiation.cc:407] Connection header received
0927 18:43:35.074038 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.074041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.074067 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.074189 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.075057 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.076560 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.077732 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.078294 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.078413 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.082669 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:35.082676 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:35.082676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:35.082683 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:35.084396 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.084794 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.084797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.084797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.084873 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.085088 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.085088 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.085089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.085303 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:35.085392 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:35.085672 (+ 280us) server_negotiation.cc:299] Negotiation successful
0927 18:43:35.085879 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":167,"thread_start_us":106,"threads_started":1}
I20260927 18:43:35.087661 2475 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:43:35.088187 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.088003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:39595 (local address 127.0.0.1:33214)
0927 18:43:35.088095 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.088157 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.253:39595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260927 18:43:35.088361 2424 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.251.253:39595: connect: Connection refused (error 111)
I20260927 18:43:35.088927 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.088820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:39595 (local address 127.0.0.1:33230)
0927 18:43:35.088875 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.088902 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.253:39595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260927 18:43:35.089129 2475 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.253:39595: Network error: Client connection negotiation failed: client connection to 127.11.251.253:39595: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.253.keytab.
Entry for principal kudu/127.11.251.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.253.keytab.
I20260927 18:43:35.101260 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.253:39595
--webserver_interface=127.11.251.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.253.keytab
--principal=kudu/127.11.251.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:44695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:43369,127.11.251.253:39595,127.11.251.252:40495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:43:35.138382 2475 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } attempt: 1
I20260927 18:43:35.139040 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.138874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:39595 (local address 127.0.0.1:33242)
0927 18:43:35.138956 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.139010 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.253:39595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260927 18:43:35.139487 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.139393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:39595 (local address 127.0.0.1:33246)
0927 18:43:35.139443 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.139471 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.253:39595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260927 18:43:35.139624 2475 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.253:39595: Network error: Client connection negotiation failed: client connection to 127.11.251.253:39595: connect: Connection refused (error 111)
W20260927 18:43:35.193910 2482 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 18:43:35.194159 2482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:35.194176 2482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:35.194198 2482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:35.195478 2482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:35.195521 2482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:35.195536 2482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:35.195550 2482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:35.195562 2482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:35.197029 2482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:44695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:43369,127.11.251.253:39595,127.11.251.252:40495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.253.keytab
--principal=kudu/127.11.251.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.253:39595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.251.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:35.197250 2482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:35.197458 2482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:35.202803 2488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:35.202800 2490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:35.202811 2487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:35.202834 2482 server_base.cc:1034] running on GCE node
I20260927 18:43:35.203364 2482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:35.203697 2482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:35.204870 2482 hybrid_clock.cc:648] HybridClock initialized: now 1790534615204854 us; error 42 us; skew 500 ppm
Sep 27 18:43:35 dist-test-slave-z47j krb5kdc[2394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534615, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:35.207571 2482 init.cc:377] Logged in from keytab as kudu/127.11.251.253@KRBTEST.COM (short username kudu)
I20260927 18:43:35.207778 2475 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } attempt: 2
I20260927 18:43:35.208223 2482 webserver.cc:460] Webserver started at http://127.11.251.253:41673/ using document root <none> and password file <none>
I20260927 18:43:35.208283 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.208112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:39595 (local address 127.0.0.1:33260)
0927 18:43:35.208195 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.208245 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.253:39595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260927 18:43:35.208525 2482 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:35.208595 2482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:35.208742 2482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:35.208781 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.208655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:39595 (local address 127.0.0.1:33274)
0927 18:43:35.208702 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.208739 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.253:39595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260927 18:43:35.208913 2475 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.253:39595: Network error: Client connection negotiation failed: client connection to 127.11.251.253:39595: connect: Connection refused (error 111)
I20260927 18:43:35.210106 2482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-1/data/instance:
uuid: "3ef162a2a04d469bbc0d3f8a08a390b2"
format_stamp: "Formatted at 2026-09-27 18:43:35 on dist-test-slave-z47j"
server_key: "7a34cf0c1227d01b3e75642010a52b1c"
server_key_iv: "bb07447f204ef8f9d09d9a378bbc54b7"
server_key_version: "encryptionkey@0"
I20260927 18:43:35.210430 2482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-1/wal/instance:
uuid: "3ef162a2a04d469bbc0d3f8a08a390b2"
format_stamp: "Formatted at 2026-09-27 18:43:35 on dist-test-slave-z47j"
server_key: "7a34cf0c1227d01b3e75642010a52b1c"
server_key_iv: "bb07447f204ef8f9d09d9a378bbc54b7"
server_key_version: "encryptionkey@0"
I20260927 18:43:35.211824 2482 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:43:35.212704 2497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:35.212926 2482 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:35.213040 2482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-1/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-1/wal
uuid: "3ef162a2a04d469bbc0d3f8a08a390b2"
format_stamp: "Formatted at 2026-09-27 18:43:35 on dist-test-slave-z47j"
server_key: "7a34cf0c1227d01b3e75642010a52b1c"
server_key_iv: "bb07447f204ef8f9d09d9a378bbc54b7"
server_key_version: "encryptionkey@0"
I20260927 18:43:35.213124 2482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:35.225299 2482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:35.227468 2482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:35.227589 2482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:35.231805 2482 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.253:39595
I20260927 18:43:35.231890 2549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.253:39595 every 8 connection(s)
I20260927 18:43:35.232232 2482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-1/data/info.pb
I20260927 18:43:35.233310 2550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:35.235251 2550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.254" port: 43369 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:43:35.237191 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 2482
I20260927 18:43:35.237308 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-1/wal/instance
I20260927 18:43:35.237568 12271 external_mini_cluster.cc:1428] Setting key 501ee526380dfa31145f4e0a3a8f0136
Sep 27 18:43:35 dist-test-slave-z47j krb5kdc[2394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534615, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.253@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.251.252@KRBTEST.COM; defaulting to no policy
I20260927 18:43:35.246588 2479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.236083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47476 (local address 127.11.251.254:43369)
0927 18:43:35.236182 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.236185 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.236394 (+ 209us) server_negotiation.cc:407] Connection header received
0927 18:43:35.236958 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.236959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.236974 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.237042 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.238071 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.238984 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.240238 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.240489 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.240580 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.243162 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:35.243163 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:35.243164 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:35.243171 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:35.244899 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.245458 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.245459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.245460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.245521 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.245785 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.245786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.245786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.245921 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:35.245997 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:35.246362 (+ 365us) server_negotiation.cc:299] Negotiation successful
0927 18:43:35.246500 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260927 18:43:35.246596 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.236038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:43369 (local address 127.0.0.1:47476)
0927 18:43:35.236233 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.236273 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:35.236840 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:35.237129 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:35.237135 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:35.237547 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:35.237954 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:35.237959 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.239101 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.239102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:35.240091 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:35.240094 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.240589 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.240590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:35.240729 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.241461 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:35.241471 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:35.243044 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:35.245106 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:35.245113 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:35.245115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:35.245363 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:35.245607 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:35.245607 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:35.245608 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:35.245717 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:35.245985 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:35.245991 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:35.246140 (+ 149us) client_negotiation.cc:769] Sending connection context
0927 18:43:35.246256 (+ 116us) client_negotiation.cc:240] Negotiation successful
0927 18:43:35.246410 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":113,"threads_started":1}
I20260927 18:43:35.247273 2550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.251.252@KRBTEST.COM" created.
Sep 27 18:43:35 dist-test-slave-z47j krb5kdc[2394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534615, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.253@KRBTEST.COM for kudu/127.11.251.253@KRBTEST.COM
I20260927 18:43:35.255115 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.247601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:39595 (local address 127.0.0.1:33286)
0927 18:43:35.247661 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.247673 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:35.247721 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:35.247963 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:35.247964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:35.248109 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:35.248159 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:35.248160 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.249089 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.249090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:35.249856 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:35.249867 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.250462 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.250463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:35.250555 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.250934 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:35.250939 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:35.252269 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:35.254058 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:35.254059 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:35.254059 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:35.254222 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:35.254437 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:35.254438 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:35.254439 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:35.254493 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:35.254774 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:35.254775 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:35.254811 (+ 36us) client_negotiation.cc:769] Sending connection context
0927 18:43:35.254890 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 18:43:35.254996 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260927 18:43:35.255187 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.247660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33286 (local address 127.11.251.253:39595)
0927 18:43:35.247818 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.247823 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.247848 (+ 25us) server_negotiation.cc:407] Connection header received
0927 18:43:35.247867 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.247869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.247880 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.247965 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.248238 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.248986 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.249967 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.250370 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.250446 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.252372 (+ 1926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:35.252379 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:35.252380 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:35.252387 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:35.253966 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.254310 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.254313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.254314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.254375 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.254557 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.254558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.254558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.254707 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:35.254799 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:35.254922 (+ 123us) server_negotiation.cc:299] Negotiation successful
0927 18:43:35.255053 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":131,"thread_start_us":75,"threads_started":1}
I20260927 18:43:35.255775 2550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:43:35.256156 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.256036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:40495 (local address 127.0.0.1:59410)
0927 18:43:35.256080 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.256132 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:40495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260927 18:43:35.256302 2501 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.251.252:40495: connect: Connection refused (error 111)
I20260927 18:43:35.256707 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.256616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:40495 (local address 127.0.0.1:59414)
0927 18:43:35.256662 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.256685 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:40495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260927 18:43:35.256820 2550 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.252:40495: Network error: Client connection negotiation failed: client connection to 127.11.251.252:40495: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.251.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.252.keytab.
Entry for principal kudu/127.11.251.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.252.keytab.
I20260927 18:43:35.264799 12271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbSvNM2/build/release/bin/kudu
/tmp/dist-test-taskbSvNM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.252:40495
--webserver_interface=127.11.251.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.252.keytab
--principal=kudu/127.11.251.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.251.212:44695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.251.254:43369,127.11.251.253:39595,127.11.251.252:40495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:43:35.300081 2475 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } attempt: 3
Sep 27 18:43:35 dist-test-slave-z47j krb5kdc[2394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534615, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for kudu/127.11.251.253@KRBTEST.COM
I20260927 18:43:35.306072 2550 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } attempt: 1
I20260927 18:43:35.306489 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.306368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:40495 (local address 127.0.0.1:59426)
0927 18:43:35.306425 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.306459 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:40495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260927 18:43:35.306969 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.306779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:40495 (local address 127.0.0.1:59442)
0927 18:43:35.306855 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.306915 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:40495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260927 18:43:35.307133 2550 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.252:40495: Network error: Client connection negotiation failed: client connection to 127.11.251.252:40495: connect: Connection refused (error 111)
I20260927 18:43:35.308769 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.300605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:39595 (local address 127.0.0.1:33290)
0927 18:43:35.300691 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.300711 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:35.300792 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:35.301032 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:35.301034 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:35.301221 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:35.301299 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:35.301301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.302364 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.302365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:35.302917 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:35.302918 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.303441 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.303442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:35.303498 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.303976 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:35.303980 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:35.305509 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:35.307745 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:35.307746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:35.307747 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:35.307917 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:35.308132 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:35.308133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:35.308133 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:35.308195 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:35.308444 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:35.308445 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:35.308492 (+ 47us) client_negotiation.cc:769] Sending connection context
0927 18:43:35.308574 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 18:43:35.308668 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260927 18:43:35.308779 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.300663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33290 (local address 127.11.251.253:39595)
0927 18:43:35.300736 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.300739 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.300784 (+ 45us) server_negotiation.cc:407] Connection header received
0927 18:43:35.300886 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.300888 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.300907 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.300986 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.301404 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.302270 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.303007 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.303361 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.303440 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.305631 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:35.305633 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:35.305634 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:35.305644 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:35.307635 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.308004 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.308005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.308006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.308066 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.308255 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.308256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.308256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.308381 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:35.308455 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:35.308597 (+ 142us) server_negotiation.cc:299] Negotiation successful
0927 18:43:35.308701 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 18:43:35.309124 2475 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:43:35.309438 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.309348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:40495 (local address 127.0.0.1:59458)
0927 18:43:35.309392 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.309417 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:40495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260927 18:43:35.309916 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.309823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:40495 (local address 127.0.0.1:59466)
0927 18:43:35.309875 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.309901 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:40495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260927 18:43:35.310026 2475 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.252:40495: Network error: Client connection negotiation failed: client connection to 127.11.251.252:40495: connect: Connection refused (error 111)
I20260927 18:43:35.341169 2475 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } attempt: 1
I20260927 18:43:35.341789 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.341621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:40495 (local address 127.0.0.1:59480)
0927 18:43:35.341692 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.341761 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:40495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260927 18:43:35.342212 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.342111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:40495 (local address 127.0.0.1:59490)
0927 18:43:35.342167 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.342196 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:40495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260927 18:43:35.342370 2475 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.252:40495: Network error: Client connection negotiation failed: client connection to 127.11.251.252:40495: connect: Connection refused (error 111)
W20260927 18:43:35.359655 2557 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 18:43:35.359915 2557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:35.359935 2557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:35.359958 2557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:35.361238 2557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:35.361279 2557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:35.361294 2557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:35.361307 2557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:35.361320 2557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:35.362934 2557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.251.212:44695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.251.254:43369,127.11.251.253:39595,127.11.251.252:40495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/krb5kdc/kudu_127.11.251.252.keytab
--principal=kudu/127.11.251.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.251.252:40495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.251.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:35.363171 2557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:35.363391 2557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:35.368726 2565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:35.368808 2563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:35.368726 2562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:35.368968 2557 server_base.cc:1034] running on GCE node
I20260927 18:43:35.369338 2557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:35.369779 2557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:35.370944 2557 hybrid_clock.cc:648] HybridClock initialized: now 1790534615370927 us; error 37 us; skew 500 ppm
Sep 27 18:43:35 dist-test-slave-z47j krb5kdc[2394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534615, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:35.373734 2557 init.cc:377] Logged in from keytab as kudu/127.11.251.252@KRBTEST.COM (short username kudu)
I20260927 18:43:35.374235 2557 webserver.cc:460] Webserver started at http://127.11.251.252:40483/ using document root <none> and password file <none>
I20260927 18:43:35.374473 2557 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:35.374521 2557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:35.374622 2557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:35.375305 2550 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } attempt: 2
I20260927 18:43:35.375785 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.375605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:40495 (local address 127.0.0.1:59494)
0927 18:43:35.375699 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.375748 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:40495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260927 18:43:35.375962 2557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-2/data/instance:
uuid: "74b7589986cf454d9f10677feb944886"
format_stamp: "Formatted at 2026-09-27 18:43:35 on dist-test-slave-z47j"
server_key: "38581be81e4495b1c824d87f6ad18589"
server_key_iv: "6b1944fb9521340fa87f678d002f776c"
server_key_version: "encryptionkey@0"
I20260927 18:43:35.376484 2557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-2/wal/instance:
uuid: "74b7589986cf454d9f10677feb944886"
format_stamp: "Formatted at 2026-09-27 18:43:35 on dist-test-slave-z47j"
server_key: "38581be81e4495b1c824d87f6ad18589"
server_key_iv: "6b1944fb9521340fa87f678d002f776c"
server_key_version: "encryptionkey@0"
I20260927 18:43:35.376566 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.376411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:40495 (local address 127.0.0.1:59500)
0927 18:43:35.376483 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.376517 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.251.252:40495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260927 18:43:35.376663 2550 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.251.252:40495: Network error: Client connection negotiation failed: client connection to 127.11.251.252:40495: connect: Connection refused (error 111)
I20260927 18:43:35.378159 2557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260927 18:43:35.379282 2572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:35.379514 2557 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:35.379618 2557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-2/data,/tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-2/wal
uuid: "74b7589986cf454d9f10677feb944886"
format_stamp: "Formatted at 2026-09-27 18:43:35 on dist-test-slave-z47j"
server_key: "38581be81e4495b1c824d87f6ad18589"
server_key_iv: "6b1944fb9521340fa87f678d002f776c"
server_key_version: "encryptionkey@0"
I20260927 18:43:35.379705 2557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:35.397845 2557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:35.400625 2557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:35.400856 2557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:35.406183 2557 rpc_server.cc:307] RPC server started. Bound to: 127.11.251.252:40495
I20260927 18:43:35.406256 2624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.251.252:40495 every 8 connection(s)
I20260927 18:43:35.406672 2557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-2/data/info.pb
I20260927 18:43:35.407678 2625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:35.409384 2625 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.254" port: 43369 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:43:35.410424 12271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbSvNM2/build/release/bin/kudu as pid 2557
I20260927 18:43:35.410578 12271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbSvNM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534329098323-12271-0/minicluster-data/master-2/wal/instance
I20260927 18:43:35.410810 12271 external_mini_cluster.cc:1428] Setting key 127231c2346ebf9be20ef25540fbafa3
Sep 27 18:43:35 dist-test-slave-z47j krb5kdc[2394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534615, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.252@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:43:35.417526 2475 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } attempt: 2
Sep 27 18:43:35 dist-test-slave-z47j krb5kdc[2394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.252@KRBTEST.COM
I20260927 18:43:35.421388 2627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.410176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.254:43369 (local address 127.0.0.1:47484)
0927 18:43:35.410365 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.410409 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:35.411041 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:35.411500 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:35.411507 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:35.411940 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:35.412225 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:35.412233 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.413247 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.413247 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:35.414409 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:35.414421 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.414944 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.414945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:35.415083 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.415774 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:35.415786 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:35.417772 (+ 1986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:35.419767 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:35.419774 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:35.419777 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:35.420078 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:35.420297 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:35.420298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:35.420299 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:35.420416 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:35.420681 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:35.420688 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:35.420851 (+ 163us) client_negotiation.cc:769] Sending connection context
0927 18:43:35.420957 (+ 106us) client_negotiation.cc:240] Negotiation successful
0927 18:43:35.421134 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":1,"thread_start_us":100,"threads_started":1}
I20260927 18:43:35.421761 2479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.410225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47484 (local address 127.11.251.254:43369)
0927 18:43:35.410331 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.410335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.410507 (+ 172us) server_negotiation.cc:407] Connection header received
0927 18:43:35.411176 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.411178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.411192 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.411297 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.412369 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.413155 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.414536 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.414830 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.414937 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.417868 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:35.417870 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:35.417871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:35.417878 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:35.419679 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.420172 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.420173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.420173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.420232 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.420480 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.420482 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.420482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.420600 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:35.420676 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:35.421538 (+ 862us) server_negotiation.cc:299] Negotiation successful
0927 18:43:35.421669 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 18:43:35.422641 2625 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:43:35 dist-test-slave-z47j krb5kdc[2394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534615, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.254@KRBTEST.COM for kudu/127.11.251.252@KRBTEST.COM
I20260927 18:43:35.422883 2629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.411296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59504 (local address 127.11.251.252:40495)
0927 18:43:35.411474 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.411477 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.411517 (+ 40us) server_negotiation.cc:407] Connection header received
0927 18:43:35.411611 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.411615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.411638 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.411718 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.412530 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.413658 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.414687 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.415195 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.415260 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.418537 (+ 3277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:35.418545 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:35.418546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:35.418553 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:35.420810 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.421183 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.421187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.421187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.421255 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.422069 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.422070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.422070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.422234 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:35.422421 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:35.422498 (+ 77us) server_negotiation.cc:299] Negotiation successful
0927 18:43:35.422596 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":156,"thread_start_us":86,"threads_started":1}
I20260927 18:43:35.427085 2630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.418063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59520 (local address 127.11.251.252:40495)
0927 18:43:35.418444 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.418446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.418453 (+ 7us) server_negotiation.cc:407] Connection header received
0927 18:43:35.418470 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.418471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.418505 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.418589 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.418923 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.419540 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.420841 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.421156 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.421464 (+ 308us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.423218 (+ 1754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:35.423220 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:35.423220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:35.423227 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:35.425488 (+ 2261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.426156 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.426157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.426157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.426202 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.426457 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.426458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.426458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.426592 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:35.426682 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:35.426878 (+ 196us) server_negotiation.cc:299] Negotiation successful
0927 18:43:35.426966 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":94,"threads_started":1}
I20260927 18:43:35.427098 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.417885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:40495 (local address 127.0.0.1:59520)
0927 18:43:35.417946 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.417960 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:35.418000 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:35.418618 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:35.418619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:35.418796 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:35.418842 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:35.418843 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.419874 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.419876 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:35.420740 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:35.420743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.421258 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.421259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:35.421326 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.421730 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:35.421735 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:35.423123 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:35.425570 (+ 2447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:35.425594 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:35.425595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:35.425842 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:35.426293 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:35.426294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:35.426294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:35.426376 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:35.426694 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:35.426695 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:35.426753 (+ 58us) client_negotiation.cc:769] Sending connection context
0927 18:43:35.426838 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 18:43:35.426948 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
Sep 27 18:43:35 dist-test-slave-z47j krb5kdc[2394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534615, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.252@KRBTEST.COM for kudu/127.11.251.253@KRBTEST.COM
I20260927 18:43:35.429033 2475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a: Bootstrap starting.
Sep 27 18:43:35 dist-test-slave-z47j krb5kdc[2394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.254@KRBTEST.COM
I20260927 18:43:35.430142 2475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:35.430475 2475 log.cc:824] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:35.432719 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.423192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33292 (local address 127.11.251.253:39595)
0927 18:43:35.423235 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.423237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.423245 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:43:35.423273 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.423274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.423284 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.423351 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.423753 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.424523 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.425489 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.425841 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.425937 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.428478 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:35.428480 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:35.428481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:35.428490 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:35.431007 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.431541 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.431559 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.431559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.431652 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.431922 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.431923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.431923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.432147 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:35.432223 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:35.432526 (+ 303us) server_negotiation.cc:299] Negotiation successful
0927 18:43:35.432623 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 18:43:35.432765 2627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.422918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.253:39595 (local address 127.0.0.1:33292)
0927 18:43:35.423158 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.423172 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:35.423219 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:35.423353 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:35.423354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:35.423541 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:35.423632 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:35.423634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.424633 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.424633 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:35.425399 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:35.425402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.425939 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.425940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:35.426028 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.426542 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:35.426547 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:35.428366 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:35.431134 (+ 2768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:35.431135 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:35.431136 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:35.431468 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:35.431714 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:35.431715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:35.431715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:35.431808 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:35.432301 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:35.432302 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:35.432351 (+ 49us) client_negotiation.cc:769] Sending connection context
0927 18:43:35.432493 (+ 142us) client_negotiation.cc:240] Negotiation successful
0927 18:43:35.432644 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222}
I20260927 18:43:35.433084 2625 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:43:35.434229 2475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a: No bootstrap required, opened a new log
I20260927 18:43:35.434557 2479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.424105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47498 (local address 127.11.251.254:43369)
0927 18:43:35.424153 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.424155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.424165 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:43:35.424187 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.424187 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.424198 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.424280 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.425203 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.425848 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.427238 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.427476 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.427562 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.430595 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:35.430596 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:35.430597 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:35.430603 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:35.432395 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.433387 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.433389 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.433389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.433496 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.433756 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.433757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.433757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.433904 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:35.434083 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:35.434380 (+ 297us) server_negotiation.cc:299] Negotiation successful
0927 18:43:35.434473 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 18:43:35.436383 2475 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dadb983ce6e438881d47ced6728242a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 43369 } } peers { permanent_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } } peers { permanent_uuid: "74b7589986cf454d9f10677feb944886" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } }
I20260927 18:43:35.436535 2475 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dadb983ce6e438881d47ced6728242a, State: Initialized, Role: FOLLOWER
I20260927 18:43:35.436640 2475 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dadb983ce6e438881d47ced6728242a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 43369 } } peers { permanent_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } } peers { permanent_uuid: "74b7589986cf454d9f10677feb944886" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } }
I20260927 18:43:35.436977 2475 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [sys.catalog]: configured and running, proceeding with master startup.
Sep 27 18:43:35 dist-test-slave-z47j krb5kdc[2394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534615, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.252@KRBTEST.COM for kudu/127.11.251.252@KRBTEST.COM
W20260927 18:43:35.439023 2643 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:43:35.439085 2643 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:43:35.441156 2632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dadb983ce6e438881d47ced6728242a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 43369 } } peers { permanent_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } } peers { permanent_uuid: "74b7589986cf454d9f10677feb944886" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } } }
I20260927 18:43:35.441241 2632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:43:35.441509 2627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.433333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:40495 (local address 127.0.0.1:59526)
0927 18:43:35.433363 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.433374 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:35.433412 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:35.433560 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:35.433561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:35.433745 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:35.433783 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:35.433785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.434629 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.434630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:35.435346 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:35.435348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.435923 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.435924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:35.435988 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.436394 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:35.436400 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:35.438087 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:35.440375 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:35.440376 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:35.440376 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:35.440584 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:35.440783 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:35.440784 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:35.440784 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:35.440834 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:35.441170 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:35.441170 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:35.441213 (+ 43us) client_negotiation.cc:769] Sending connection context
0927 18:43:35.441278 (+ 65us) client_negotiation.cc:240] Negotiation successful
0927 18:43:35.441417 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260927 18:43:35.442111 2630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.433401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59526 (local address 127.11.251.252:40495)
0927 18:43:35.433468 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.433469 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.433474 (+ 5us) server_negotiation.cc:407] Connection header received
0927 18:43:35.433495 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.433495 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.433502 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.433557 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.433870 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.434542 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.435443 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.435789 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.435871 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.438183 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:35.438184 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:35.438185 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:35.438192 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:35.440244 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.440664 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.440665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.440665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.440719 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.440893 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.440894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.440894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.441082 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:35.441145 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:35.441943 (+ 798us) server_negotiation.cc:299] Negotiation successful
0927 18:43:35.442029 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 18:43:35.444218 2625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74b7589986cf454d9f10677feb944886: Bootstrap starting.
Sep 27 18:43:35 dist-test-slave-z47j krb5kdc[2394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.251.253@KRBTEST.COM
I20260927 18:43:35.445096 2625 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74b7589986cf454d9f10677feb944886: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:35.445433 2625 log.cc:824] T 00000000000000000000000000000000 P 74b7589986cf454d9f10677feb944886: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:35.441818 2407 master_runner.cc:186] Error getting master registration for 127.11.251.253:39595: OK, instance_id { permanent_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2" instance_seqno: 1790534615230603 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 18:43:35.445729 2407 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 18:43:35.448457 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.436354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33300 (local address 127.11.251.253:39595)
0927 18:43:35.437730 (+ 1376us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.437732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.437745 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:43:35.437769 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.437769 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.437780 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.437956 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.439884 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.440607 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.442100 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.442399 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.442447 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.445519 (+ 3072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:35.445520 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:35.445521 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:35.445527 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:35.447246 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.447685 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.447686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.447686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.447745 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.447945 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.447946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.447946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.448084 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:35.448251 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:35.448293 (+ 42us) server_negotiation.cc:299] Negotiation successful
0927 18:43:35.448369 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1350}
I20260927 18:43:35.450129 12271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:43:35.450400 2625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74b7589986cf454d9f10677feb944886: No bootstrap required, opened a new log
I20260927 18:43:35.451510 2449 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0dadb983ce6e438881d47ced6728242a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47498
I20260927 18:43:35.451692 2449 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260927 18:43:35.451742 2449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:35.453086 2449 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dadb983ce6e438881d47ced6728242a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 43369 } } peers { permanent_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } } peers { permanent_uuid: "74b7589986cf454d9f10677feb944886" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } }
I20260927 18:43:35.453675 2449 leader_election.cc:288] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [CANDIDATE]: Term 1 election: Requested vote from peers 3ef162a2a04d469bbc0d3f8a08a390b2 (127.11.251.253:39595), 74b7589986cf454d9f10677feb944886 (127.11.251.252:40495)
I20260927 18:43:35.454810 2625 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74b7589986cf454d9f10677feb944886 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dadb983ce6e438881d47ced6728242a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 43369 } } peers { permanent_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } } peers { permanent_uuid: "74b7589986cf454d9f10677feb944886" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } }
I20260927 18:43:35.455304 2625 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74b7589986cf454d9f10677feb944886 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74b7589986cf454d9f10677feb944886, State: Initialized, Role: FOLLOWER
I20260927 18:43:35.455453 2625 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74b7589986cf454d9f10677feb944886 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dadb983ce6e438881d47ced6728242a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 43369 } } peers { permanent_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } } peers { permanent_uuid: "74b7589986cf454d9f10677feb944886" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } }
I20260927 18:43:35.455901 2625 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74b7589986cf454d9f10677feb944886 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:35.455943 2599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0dadb983ce6e438881d47ced6728242a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "74b7589986cf454d9f10677feb944886"
I20260927 18:43:35.456441 2599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74b7589986cf454d9f10677feb944886 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:35.457862 2599 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 74b7589986cf454d9f10677feb944886 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dadb983ce6e438881d47ced6728242a in term 1.
I20260927 18:43:35.458204 2426 leader_election.cc:302] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dadb983ce6e438881d47ced6728242a, 74b7589986cf454d9f10677feb944886; no voters:
I20260927 18:43:35.458397 2632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:35.458578 2632 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [term 1 LEADER]: Becoming Leader. State: Replica: 0dadb983ce6e438881d47ced6728242a, State: Running, Role: LEADER
I20260927 18:43:35.458705 2632 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dadb983ce6e438881d47ced6728242a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 43369 } } peers { permanent_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } } peers { permanent_uuid: "74b7589986cf454d9f10677feb944886" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } }
I20260927 18:43:35.459263 2632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dadb983ce6e438881d47ced6728242a. Latest consensus state: current_term: 1 leader_uuid: "0dadb983ce6e438881d47ced6728242a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dadb983ce6e438881d47ced6728242a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 43369 } } peers { permanent_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } } peers { permanent_uuid: "74b7589986cf454d9f10677feb944886" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } } }
I20260927 18:43:35.459445 2632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:35.457784 2650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74b7589986cf454d9f10677feb944886 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dadb983ce6e438881d47ced6728242a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 43369 } } peers { permanent_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } } peers { permanent_uuid: "74b7589986cf454d9f10677feb944886" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } } }
I20260927 18:43:35.461172 2650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74b7589986cf454d9f10677feb944886 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:43:35.463950 2666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:35.464627 2666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:35.462648 2557 master_runner.cc:186] Error getting master registration for 127.11.251.253:39595: OK, instance_id { permanent_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2" instance_seqno: 1790534615230603 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 18:43:35.466859 2557 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 18:43:35.467343 2479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.455698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47508 (local address 127.11.251.254:43369)
0927 18:43:35.455749 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.455751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.458111 (+ 2360us) server_negotiation.cc:407] Connection header received
0927 18:43:35.458170 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.458171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.458183 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.458239 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.459214 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.459972 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.461384 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.461714 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.461759 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.464236 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:35.464237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:35.464238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:35.464244 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:35.466026 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.466419 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.466421 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.466421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.466473 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.466681 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.466682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.466682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.466809 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:35.466891 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:35.467175 (+ 284us) server_negotiation.cc:299] Negotiation successful
0927 18:43:35.467254 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:43:35.467808 2550 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } attempt: 3
I20260927 18:43:35.468266 2630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.456035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59538 (local address 127.11.251.252:40495)
0927 18:43:35.456084 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.456087 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.458247 (+ 2160us) server_negotiation.cc:407] Connection header received
0927 18:43:35.458389 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.458390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.458401 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.458433 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.459410 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.460209 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.461414 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.461721 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.461765 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.464111 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:35.464113 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:35.464113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:35.464121 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:35.466618 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.467103 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.467104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.467105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.467179 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.467491 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.467493 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.467493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.467638 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:35.467741 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:35.468057 (+ 316us) server_negotiation.cc:299] Negotiation successful
0927 18:43:35.468142 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260927 18:43:35.470114 2667 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 74b7589986cf454d9f10677feb944886: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:43:35.470185 2667 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:43:35.470546 2599 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 74b7589986cf454d9f10677feb944886 [term 1 FOLLOWER]: Refusing update from remote peer 0dadb983ce6e438881d47ced6728242a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:35.470992 2664 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [LEADER]: Connected to new peer: Peer: permanent_uuid: "74b7589986cf454d9f10677feb944886" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:35.472625 2650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74b7589986cf454d9f10677feb944886 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dadb983ce6e438881d47ced6728242a. Latest consensus state: current_term: 1 leader_uuid: "0dadb983ce6e438881d47ced6728242a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dadb983ce6e438881d47ced6728242a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 43369 } } peers { permanent_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } } peers { permanent_uuid: "74b7589986cf454d9f10677feb944886" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } } }
I20260927 18:43:35.472718 2650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74b7589986cf454d9f10677feb944886 [sys.catalog]: This master's current role is: FOLLOWER
Sep 27 18:43:35 dist-test-slave-z47j krb5kdc[2394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534615, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.251.253@KRBTEST.COM for kudu/127.11.251.252@KRBTEST.COM
I20260927 18:43:35.474202 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.456449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33308 (local address 127.11.251.253:39595)
0927 18:43:35.457740 (+ 1291us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.457742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.462218 (+ 4476us) server_negotiation.cc:407] Connection header received
0927 18:43:35.462457 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.462458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.462471 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.462501 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.465087 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.465888 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.468249 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.468567 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.468610 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.470734 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:35.470736 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:35.470736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:35.470744 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:35.472744 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.473179 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.473180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.473181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.473244 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.473468 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.473468 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.473469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.473629 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:35.473731 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:35.473989 (+ 258us) server_negotiation.cc:299] Negotiation successful
0927 18:43:35.474079 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1256}
I20260927 18:43:35.476562 2630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.468306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59554 (local address 127.11.251.252:40495)
0927 18:43:35.468345 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.468347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.468367 (+ 20us) server_negotiation.cc:407] Connection header received
0927 18:43:35.468385 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.468386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.468395 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.468456 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.468787 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.469459 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.470869 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.471165 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.471228 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.473424 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:35.473426 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:35.473426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:35.473434 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:35.475346 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.475734 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.475735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.475736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.475793 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:35.476012 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:35.476013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:35.476013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:35.476156 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:35.476218 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:35.476395 (+ 177us) server_negotiation.cc:299] Negotiation successful
0927 18:43:35.476467 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:43:35.476563 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:35.468087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.251.252:40495 (local address 127.0.0.1:59554)
0927 18:43:35.468175 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:35.468187 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:35.468225 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:35.468465 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:35.468466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:35.468629 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:35.468686 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:35.468688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.469553 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.469554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:35.470772 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:35.470775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:35.471282 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:35.471283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:35.471338 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:35.471726 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:35.471732 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:35.473316 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:35.475422 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:35.475423 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:35.475424 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:35.475648 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:35.475872 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:35.475873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:35.475874 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:35.475939 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:35.476223 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:35.476223 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:35.476267 (+ 44us) client_negotiation.cc:769] Sending connection context
0927 18:43:35.476349 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 18:43:35.476467 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260927 18:43:35.478438 2550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ef162a2a04d469bbc0d3f8a08a390b2: Bootstrap starting.
I20260927 18:43:35.478755 2650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74b7589986cf454d9f10677feb944886 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0dadb983ce6e438881d47ced6728242a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dadb983ce6e438881d47ced6728242a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 43369 } } peers { permanent_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } } peers { permanent_uuid: "74b7589986cf454d9f10677feb944886" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } } }
I20260927 18:43:35.478827 2650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74b7589986cf454d9f10677feb944886 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:43:35.478781 2632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0dadb983ce6e438881d47ced6728242a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dadb983ce6e438881d47ced6728242a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 43369 } } peers { permanent_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } } peers { permanent_uuid: "74b7589986cf454d9f10677feb944886" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } } }
I20260927 18:43:35.479074 2632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:35.479316 2670 mvcc.cc:205] Tried to move back new op lower bound from 7334029784961458176 to 7334029784919953408. Current Snapshot: MvccSnapshot[applied={T|T < 7334029784961458176}]
I20260927 18:43:35.479398 2550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ef162a2a04d469bbc0d3f8a08a390b2: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:35.479686 2550 log.cc:824] T 00000000000000000000000000000000 P 3ef162a2a04d469bbc0d3f8a08a390b2: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:35.481071 2550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ef162a2a04d469bbc0d3f8a08a390b2: No bootstrap required, opened a new log
I20260927 18:43:35.483197 2550 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ef162a2a04d469bbc0d3f8a08a390b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dadb983ce6e438881d47ced6728242a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 43369 } } peers { permanent_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } } peers { permanent_uuid: "74b7589986cf454d9f10677feb944886" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } }
I20260927 18:43:35.483349 2550 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ef162a2a04d469bbc0d3f8a08a390b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef162a2a04d469bbc0d3f8a08a390b2, State: Initialized, Role: FOLLOWER
I20260927 18:43:35.483443 2550 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ef162a2a04d469bbc0d3f8a08a390b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dadb983ce6e438881d47ced6728242a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 43369 } } peers { permanent_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } } peers { permanent_uuid: "74b7589986cf454d9f10677feb944886" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } }
I20260927 18:43:35.483790 2550 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ef162a2a04d469bbc0d3f8a08a390b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:35.483760 2676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ef162a2a04d469bbc0d3f8a08a390b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dadb983ce6e438881d47ced6728242a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 43369 } } peers { permanent_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } } peers { permanent_uuid: "74b7589986cf454d9f10677feb944886" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } } }
I20260927 18:43:35.483841 2524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0dadb983ce6e438881d47ced6728242a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2"
I20260927 18:43:35.484056 2523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ef162a2a04d469bbc0d3f8a08a390b2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:35.484248 2676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ef162a2a04d469bbc0d3f8a08a390b2 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:43:35.485180 2523 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3ef162a2a04d469bbc0d3f8a08a390b2 [term 1 FOLLOWER]: Refusing update from remote peer 0dadb983ce6e438881d47ced6728242a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:35.485538 2632 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:35.486102 2666 catalog_manager.cc:1347] Generated new cluster ID: 8fac4588588c4095832b4182387032e4
I20260927 18:43:35.486155 2666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:35.486219 2524 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3ef162a2a04d469bbc0d3f8a08a390b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dadb983ce6e438881d47ced6728242a in term 1.
I20260927 18:43:35.487689 2676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ef162a2a04d469bbc0d3f8a08a390b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dadb983ce6e438881d47ced6728242a. Latest consensus state: current_term: 1 leader_uuid: "0dadb983ce6e438881d47ced6728242a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dadb983ce6e438881d47ced6728242a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 43369 } } peers { permanent_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } } peers { permanent_uuid: "74b7589986cf454d9f10677feb944886" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } } }
I20260927 18:43:35.487780 2676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ef162a2a04d469bbc0d3f8a08a390b2 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:43:35.487823 2676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ef162a2a04d469bbc0d3f8a08a390b2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0dadb983ce6e438881d47ced6728242a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dadb983ce6e438881d47ced6728242a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 43369 } } peers { permanent_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } } peers { permanent_uuid: "74b7589986cf454d9f10677feb944886" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } } }
I20260927 18:43:35.487887 2676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ef162a2a04d469bbc0d3f8a08a390b2 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:43:35.497761 2666 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:35.498486 2666 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:35.498751 2632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0dadb983ce6e438881d47ced6728242a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dadb983ce6e438881d47ced6728242a" member_type: VOTER last_known_addr { host: "127.11.251.254" port: 43369 } } peers { permanent_uuid: "3ef162a2a04d469bbc0d3f8a08a390b2" member_type: VOTER last_known_addr { host: "127.11.251.253" port: 39595 } } peers { permanent_uuid: "74b7589986cf454d9f10677feb944886" member_type: VOTER last_known_addr { host: "127.11.251.252" port: 40495 } } }
I20260927 18:43:35.498872 2632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a [sys.catalog]: This master's current role is: LEADER
W20260927 18:43:35.500140 2690 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ef162a2a04d469bbc0d3f8a08a390b2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:43:35.500203 2690 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:43:35.500485 2687 mvcc.cc:205] Tried to move back new op lower bound from 7334029784961458176 to 7334029784919953408. Current Snapshot: MvccSnapshot[applied={T|T < 7334029784961458176}]
I20260927 18:43:35.502709 2666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0dadb983ce6e438881d47ced6728242a: Generated new TSK 0
I20260927 18:43:35.502875 2666 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 18:43:35.530217 2630 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:35.528523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59558 (local address 127.11.251.252:40495)
0927 18:43:35.528572 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.528574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.529492 (+ 918us) server_negotiation.cc:407] Connection header received
0927 18:43:35.529636 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.529637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.529649 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.529681 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.530153 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59558: BlockingRecv error: recv got EOF from 127.0.0.1:59558 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260927 18:43:35.530951 2479 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:35.528884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47512 (local address 127.11.251.254:43369)
0927 18:43:35.528931 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.528933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.530464 (+ 1531us) server_negotiation.cc:407] Connection header received
0927 18:43:35.530559 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.530560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.530572 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.530602 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.530891 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47512: BlockingRecv error: recv got EOF from 127.0.0.1:47512 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260927 18:43:35.531754 2554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:35.529219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33324 (local address 127.11.251.253:39595)
0927 18:43:35.529261 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.529263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.531221 (+ 1958us) server_negotiation.cc:407] Connection header received
0927 18:43:35.531330 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.531331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.531341 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.531370 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.531696 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33324: BlockingRecv error: recv got EOF from 127.0.0.1:33324 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260927 18:43:35.538326 2554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:35.534182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33326 (local address 127.11.251.253:39595)
0927 18:43:35.534216 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.534218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.537778 (+ 3560us) server_negotiation.cc:407] Connection header received
0927 18:43:35.537924 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.537926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.537944 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.537972 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.538281 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33326: BlockingRecv error: recv got EOF from 127.0.0.1:33326 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260927 18:43:35.541345 2554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:35.540631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33328 (local address 127.11.251.253:39595)
0927 18:43:35.540662 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.540663 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.540876 (+ 213us) server_negotiation.cc:407] Connection header received
0927 18:43:35.540965 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.540966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.540975 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.541002 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.541298 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33328: BlockingRecv error: recv got EOF from 127.0.0.1:33328 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260927 18:43:35.545145 2554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:35.544448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33336 (local address 127.11.251.253:39595)
0927 18:43:35.544481 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.544482 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.544690 (+ 208us) server_negotiation.cc:407] Connection header received
0927 18:43:35.544780 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.544781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.544790 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.544817 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.545105 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33336: BlockingRecv error: recv got EOF from 127.0.0.1:33336 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260927 18:43:35.550938 2554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:35.550237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33344 (local address 127.11.251.253:39595)
0927 18:43:35.550270 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.550271 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.550481 (+ 210us) server_negotiation.cc:407] Connection header received
0927 18:43:35.550571 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.550572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.550581 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.550609 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.550897 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33344: BlockingRecv error: recv got EOF from 127.0.0.1:33344 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260927 18:43:35.560835 2554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:35.560082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33356 (local address 127.11.251.253:39595)
0927 18:43:35.560122 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.560124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.560345 (+ 221us) server_negotiation.cc:407] Connection header received
0927 18:43:35.560439 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.560439 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.560449 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.560477 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.560788 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33356: BlockingRecv error: recv got EOF from 127.0.0.1:33356 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260927 18:43:35.579416 2554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:35.578267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33362 (local address 127.11.251.253:39595)
0927 18:43:35.578313 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.578315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.578583 (+ 268us) server_negotiation.cc:407] Connection header received
0927 18:43:35.578697 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.578698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.578711 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.578762 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.579355 (+ 593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33362: BlockingRecv error: recv got EOF from 127.0.0.1:33362 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260927 18:43:35.613899 2554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:35.612878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33372 (local address 127.11.251.253:39595)
0927 18:43:35.612931 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.612934 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.613226 (+ 292us) server_negotiation.cc:407] Connection header received
0927 18:43:35.613340 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.613342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.613355 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.613386 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.613833 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33372: BlockingRecv error: recv got EOF from 127.0.0.1:33372 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260927 18:43:35.680456 2554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:35.679398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33384 (local address 127.11.251.253:39595)
0927 18:43:35.679487 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.679491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.679618 (+ 127us) server_negotiation.cc:407] Connection header received
0927 18:43:35.679728 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.679731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.679753 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.679870 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.680354 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33384: BlockingRecv error: recv got EOF from 127.0.0.1:33384 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260927 18:43:35.810629 2554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:35.809635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33398 (local address 127.11.251.253:39595)
0927 18:43:35.809727 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:35.809729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:35.809964 (+ 235us) server_negotiation.cc:407] Connection header received
0927 18:43:35.810008 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:35.810009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:35.810025 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:35.810080 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:35.810531 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33398: BlockingRecv error: recv got EOF from 127.0.0.1:33398 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260927 18:43:36.068915 2554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:36.068057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33404 (local address 127.11.251.253:39595)
0927 18:43:36.068127 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:36.068130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:36.068268 (+ 138us) server_negotiation.cc:407] Connection header received
0927 18:43:36.068380 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:36.068382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:36.068398 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:36.068462 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:36.068835 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33404: BlockingRecv error: recv got EOF from 127.0.0.1:33404 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:43:36.470687 2667 catalog_manager.cc:1259] Loaded cluster ID: 8fac4588588c4095832b4182387032e4
I20260927 18:43:36.470772 2667 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 74b7589986cf454d9f10677feb944886: loading cluster ID for follower catalog manager: success
I20260927 18:43:36.471647 2667 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 74b7589986cf454d9f10677feb944886: acquiring CA information for follower catalog manager: success
I20260927 18:43:36.471982 2667 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 74b7589986cf454d9f10677feb944886: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 18:43:36.500840 2690 catalog_manager.cc:1259] Loaded cluster ID: 8fac4588588c4095832b4182387032e4
I20260927 18:43:36.500945 2690 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3ef162a2a04d469bbc0d3f8a08a390b2: loading cluster ID for follower catalog manager: success
I20260927 18:43:36.501883 2690 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3ef162a2a04d469bbc0d3f8a08a390b2: acquiring CA information for follower catalog manager: success
I20260927 18:43:36.502187 2690 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3ef162a2a04d469bbc0d3f8a08a390b2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 18:43:36.585958 2759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:36.582725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33420 (local address 127.11.251.253:39595)
0927 18:43:36.582924 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:36.582926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:36.582959 (+ 33us) server_negotiation.cc:407] Connection header received
0927 18:43:36.583065 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:36.583065 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:36.583185 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:36.583282 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:36.583552 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:36.584064 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:36.585058 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:36.585264 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:36.585318 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:36.585503 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:36.585710 (+ 207us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:36.585778 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:36.585832 (+ 54us) server_negotiation.cc:299] Negotiation successful
0927 18:43:36.585851 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":116,"threads_started":1}
I20260927 18:43:36.590765 2765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:36.588143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59560 (local address 127.11.251.252:40495)
0927 18:43:36.588267 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:36.588269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:36.588277 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:43:36.588323 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:36.588324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:36.588368 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:36.588453 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:36.588629 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:36.589171 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:36.590016 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:36.590213 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:36.590275 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:36.590418 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:36.590556 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:36.590621 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:36.590668 (+ 47us) server_negotiation.cc:299] Negotiation successful
0927 18:43:36.590685 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260927 18:43:36.595815 2759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:36.592608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33422 (local address 127.11.251.253:39595)
0927 18:43:36.592660 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:36.592662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:36.592926 (+ 264us) server_negotiation.cc:407] Connection header received
0927 18:43:36.593012 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:36.593013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:36.593027 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:36.593094 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:36.593263 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:36.594025 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:36.594997 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:36.595317 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:36.595379 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:36.595564 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:36.595597 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:36.595640 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:36.595699 (+ 59us) server_negotiation.cc:299] Negotiation successful
0927 18:43:36.595728 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 18:43:36.596125 2765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:36.592683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59568 (local address 127.11.251.252:40495)
0927 18:43:36.592766 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:36.592769 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:36.593120 (+ 351us) server_negotiation.cc:407] Connection header received
0927 18:43:36.593206 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:36.593207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:36.593219 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:36.593262 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:36.593437 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:36.593988 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:36.595151 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:36.595513 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:36.595588 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:36.595760 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:36.595799 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:36.595852 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:36.595998 (+ 146us) server_negotiation.cc:299] Negotiation successful
0927 18:43:36.596027 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260927 18:43:36.596630 2770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:36.592608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47520 (local address 127.11.251.254:43369)
0927 18:43:36.592902 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:36.592905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:36.592940 (+ 35us) server_negotiation.cc:407] Connection header received
0927 18:43:36.593052 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:36.593053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:36.593108 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:36.593197 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:36.593393 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:36.594173 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:36.595260 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:36.595517 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:36.595616 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:36.596042 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:36.596159 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:36.596238 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:36.596543 (+ 305us) server_negotiation.cc:299] Negotiation successful
0927 18:43:36.596570 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":61,"threads_started":1}
I20260927 18:43:36.598793 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 2407
I20260927 18:43:36.604882 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 2482
I20260927 18:43:36.611348 12271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbSvNM2/build/release/bin/kudu with pid 2557
2026-09-27T18:43:36Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2852 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5828 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (287519 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST